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ABSTRACT 


By  using  visualization  best  practices  and  embedding  them  in  information 
technology  (IT),  we  believe  that  the  Department  of  Defense  can  improve  its  ability  to 
display  multi-variant  information. 

The  focus  of  this  research  is  to  design  a  visual  information  solution,  based  on  best 
practices  for  displaying  performance  data  visually,  to  the  Electronic  Military  Personnel 
Record  System  (EMPRS).  Ultimately,  our  goal  is  to  improve  the  overall  effectiveness 
and  objectivity  of  the  Navy’s  selection  board  processes  by  providing  a  re-engineered, 
web-based,  graphical  solution  to  the  visual  displays  currently  in  use  by  selection  boards. 

The  current  Navy  selection  board  voting  process  uses  tabular  forms  displayed 
across  five  screens  in  a  small  theater-like  setting.  The  forms  are  displayed  very  quickly 
allowing  board  members  very  little  time  to  mentally  assimilate  the  quantitative  data 
dispersed  over  a  wide  area.  In  our  model,  we  distill  the  data  into  a  single  graphical 
display,  thus  reducing  the  cognitive  computing  requirements  of  the  board  members. 

We  used  the  Knowledge  Value  Added  methodology  to  determine  the  proposal’s 
relative  effectiveness  and  developed  a  prototype  as  a  proof  of  concept.  With  this  study 
and  follow  on  recommendations,  we  foresee  a  considerable  improvement  potential  in  the 
Navy’s  promotion  board  procedures  and  outcomes. 
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EXECUTIVE  SUMMARY 


A  critical  issue  in  an  officer’s  career  is  the  opportunity  for  promotion.  In  today’s 
technology  information  rich  environment,  it  is  important  to  re-evaluate  and  possibly 
change  the  way  promotion  infonnation  is  presented  to  a  selection  board.  The  current 
method  of  displaying  a  candidate’s  personnel  record  for  promotion  or  selection  relies 
heavily  on  human  cognition  to  process  tabular,  quantitative  information  in  the  form  of  the 
Officer  Summary  Record  (OSR)  and  the  Performance  Summary  Record  (PSR) 
information.  Board  members  use  the  OSR/PSR  infonnation  to  gain  insight  into  the 
candidate’s  personnel  data,  promotion  history,  educational  background,  and  job 
performance  in  specific  billets  to  evaluate  each  candidate  for  promotion  potential.  The 
current  data-to-information  conversion  process  relies  entirely  on  notes,  symbols,  and 
highlighted  annotations  on  the  digital  summary  record  forms  to  convey  the  candidate’s 
potential  for  promotion  to  the  other  board  members.  During  the  promotion  process,  this 
quantitative  infonnation  is  displayed  across  three  to  five  screens  in  a  small  theater-like 
setting.  The  forms  are  displayed  for  two  to  three  minutes  as  the  individual’s  record  is 
reviewed  by  the  board  members.  This  restricted  form  of  information  presentation  makes 
it  difficult  to  compare  candidates  and  may  even  lead  to  distortions  in  the  evaluation 
process. 

We  consider  it  essential  that  the  board  members  can  quickly  review  and 
understand  the  information  presented  on  the  summary  forms;  information  currently 
extracted  from  several  legacy  data  systems  using  outdated  infonnation  retrieval 
processes.  We  see  a  short-lived  opportunity  as  the  Navy  plans  to  move  toward  a  single 
source  human  resource  system  and  concurrently  redesigns  the  Electronic  Military 
Personnel  Record  System  (EMPRS).  Now  is  an  appropriate  time  to  review  the  adequacy 
of  the  tools  used  to  present  career  information  and  if  is  the  system  developers  determine 
that  there  is  a  more  useful  career  information  summary  tool  for  selection  board  members, 
the  tool  needs  to  be  identified  early  in  the  development  of  future  information  systems  and 
integrated  into  the  proposed  designs. 
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One  of  the  greatest  benefits  of  data  visualization  is  the  sheer  quantity  of 
information  that  can  be  rapidly  interpreted  if  presented  well  (Ware,  2000).  This  thesis 
will  offer  examples  of  how  to  best  transform  the  data  into  information  that  the  board 
members  can  use  for  optimal  decision  making.  Our  goal  is  to  provide  a  re-engineered, 
web-based,  graphical  solution  to  the  visual  displays  currently  used  by  selection  boards. 
The  improved  format  will  display  an  individual’s  career  data  by  graphically  displaying  an 
officer’s  career  data  in  a  method  that  maximizes  infonnation  flow  to  the  briefer  and 
selection  board  while  minimizing  selection  board  member  training,  record  preparation 
time,  and  mental  “data  to  information”  processing  time  in  the  promotion  evaluation 
process. 

This  study  includes  an  in  depth  literature  review  of  the  most  current  visualization 
techniques  for  multi-criteria  decision-making  models  in  conjunction  with  well  established 
principles  for  the  display  of  visual  information.  Much  of  this  theory  is  based  off  of 
Edward  Tufte’s  classical  works  on  the  visualization  of  data. 

The  Knowledge  Value  Added  (KVA)  methodology  is  used  in  order  to  measure 
the  relative  effectiveness  of  the  proposal  against  the  current  system.  KVA  is  a  framework 
for  measuring  the  value  of  organizations  knowledge  assets  and  provides  a  methodology 
for  allocating  revenue  and  cost  to  an  organization’s  core  processes,  based  on  the  amount 
of  change  each  produces.  KVA  standardizes  the  output  of  all  processes  by  describing  the 
output  in  terms  of  the  units  of  knowledge  required  to  produce  it.  KVA  uses  a  Return  on 
Knowledge  (ROK)  ratio  of  the  current  system  compared  to  the  proposed  system  in  order 
to  compare  the  measurement  of  the  knowledge  value  embedded  in  the  organization’s  core 
processes. 

Our  main  research  question  asks  whether  data-information  visualization  best 
practices  can  be  embedded  in  information  technology  in  order  to  improve  the  overall 
effectiveness  and  objectiveness  of  the  selection  board  process.  We  introduce  a  proof-of- 
concept  (POC)  prototype  visualization  solution  through  the  use  of  Commercial  Off  The 
Shelf  (COTS)  software  and  technology  in  order  to  demonstrate  the  visualization  concepts 
and  capabilities  currently  available. 
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The  KVA  analysis  we  calculated  in  this  study  proves  that  by  using  a  web-enabled 
system  and  graphically  displaying  the  information,  the  process  becomes  more  efficient  by 
saving  time  and  money  during  each  board  session.  For  the  next  phase  of  future 
evaluation,  we  recommend  that  a  side-by-side  analysis  using  a  beta  system  be  used  in 
conjunction  with  the  old  system  in  an  actual  board.  By  surveying  the  board  members 
immediately  after  the  board  session,  researchers  could  harvest  immediate  insights 
whether  or  not  the  new  graphical  displays  assist  in  perceiving  and  correlating  the 
quantitative  information.  Our  study  suggests  that  by  surveying  multiple  boards  and 
making  the  recommended  improvements,  future  researchers  could  acquire  empirical 
feedback  necessary  to  determine  whether  or  not  the  proposed  system  contributes  or 
detracts  from  increasing  the  effectiveness  and  objectiveness  of  the  overall  process.  This 
thesis  also  suggests  that  by  reducing  the  preparation,  briefing,  and  voting  times  for  the 
board  members,  some  of  the  emotional  and  physical  strain  internal  to  multiple  decision¬ 
making  within  a  confined  time  period  would  also  be  reduced.  This  improvement  may 
well  translate  to  more  effective,  and  ultimately  more  objective  decisions. 

In  summary,  this  thesis’  primary  practical  focus  is  to  make  recommendations  that 
will  enhance  the  board  members’  comprehension  of  the  data  which  in  turn,  renders  the 
current  process  more  effective,  efficient,  and  objective.  The  recommendations  set  forth 
in  this  study  are  important  concepts  that  may  contribute  to  immediate,  short-term  results 
such  as  shortening  the  selection  board  process  while  increasing  its  efficiency.  A 
significant,  long  term  effect  includes  a  potential  increase  in  retention  rates  resulting  from 
a  perceived  improvement  and  effectiveness  in  the  promotion  or  assignment  systems. 
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I.  INTRODUCTION 


A.  PURPOSE 

With  the  advancement  of  technology  and  in  particular  the  rapid  progression  of  the 
Internet,  decision  makers  face  an  increasing  problem  of  having  too  much  raw  infonnation 
available.  Often  this  problem  is  further  exacerbated  by  the  fact  that  all  of  the  data  and 
multi-variant  infonnation  is  not  displayed  in  the  most  effective  manner.  As  the  speed  and 
complexity  of  information  greatly  increases,  the  demand  for  visual  tools  to  make  sense  of 
that  information  will  only  grow.  By  using  visualization  best  practices  and  embedding 
them  in  information  technology  (IT),  we  believe  that  the  Department  of  Defense  can 
improve  its  ability  to  display  multi-variant  information.  This  thesis’s  goal  is  to  determine 
the  best  way  to  communicate  more  information  per  unit  of  time.  In  other  words,  what  is 
the  best  way  to  transform  data  into  infonnation  that  can  be  quickly  and  accurately 
communicated  effectively  to  others? 

The  focus  of  this  research  is  to  design  a  visual  information  solution,  based  on 
best  practices  for  displaying  performance  data  visually,  to  the  Electronic  Military 
Personnel  Record  System  (EMPRS)  to  improve  the  overall  effectiveness  and  objectivity 
of  the  Navy’s  selection  board  processes.  The  goal  is  to  provide  a  re-engineered,  web- 
based,  graphical  solution  to  the  visual  displays  currently  in  use  by  selection  boards,  in  an 
improved  manner  and  format  that  will  display  an  individual’s  career  data.  By  maximizing 
information  flow  to  the  briefer  and  selection  board  and  minimizing  selection  board 
member  training,  record  preparation  time,  and  labor-intensive  “data  to  information” 
processing  time  in  the  promotion  evaluation  process,  the  ultimate  re-design  should  be 
clear,  concise,  comprehensive,  and  entail  a  scalable  methodology  allowing  for  changing 
performance  dimensions.  An  additional  objective  is  to  propose  the  Knowledge  Value 
Added  (KVA)  methodology  as  a  metric  by  which  to  gauge  or  measure  this  solution 
against  the  current  process  in  order  to  determine  its  relative  future  effectiveness. 

1.  Scope 

This  thesis  is  primarily  concerned  with  the  graphical  representation  of  information 
to  maximize  information  flow  about  an  individual’s  promotion/selection  potential  while 
minimizing  the  amount  of  effort  the  user  expends  extracting  this  infonnation.  Emphasis 
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is  placed  on  the  evaluation  of  the  human  factors  involved  in  the  effective  display  of 
visual  information,  to  include  size  of  display,  colors  used  in  the  display,  and  effective 
ways  to  present  infonnation.  The  current  form  uses  only  text  and  grid  lines.  Included  in 
this  study  is  an  evaluation  of  other  ways  to  present  information,  such  as  with  charts  or 
graphs,  to  more  effectively  convey  the  information  reviewed  on  the  summary  form. 

The  vision  of  this  project  as  articulated  by  the  Navy  Personnel  Command  (NPC) 
is  that  of  a  two-phased  implementation  with  the  first  phase  being  the  incremental  step  of 
re-designing  the  visual  displays  to  look  at  today’s  “Report  Card”  (OSR/PSR  data).  This  is 
the  phase  that  our  thesis  addresses.  The  second  phase  is  a  radical  re-design  of  the  displays 
to  encompass  “Tomorrow’s  Five  Vector  Model”  in  the  advancement  and  future 
promotion  of  Naval  personnel.  The  scope  of  the  two  phased  implementation  is  as  follows: 

a.  Naval  Postgraduate  School  (NPS)  Project  Scope 

The  current  displays  are  used  by  selection  boards,  detailers,  and  other 
career  planners  to  determine  the  future  promotion,  advancement,  and  assignment  of  Navy 
personnel.  In  an  attempt  to  limit  the  scope  of  this  project,  the  authors  have  decided  only 
to  use  the  officer  statutory  selection  board  process  example  in  which  to  base  the  study 
and  proof  of  concept  model.  Our  intent  is  that  this  system  will  evolve  into  a  true  web- 
based  interactive  model,  however,  at  the  present  time  this  web-based  format  has  yet  to  be 
identified,  so  we  have  chosen  to  develop  a  proof  of  concept  model  using  Macromedia 
Dreamweaver  MX  for  our  web  application,  utilizing  a  three  tier  architecture 
encompassing  a  notional  Microsoft  Access  relational  database.  Due  to  the  limited  time 
and  resources  available  for  this  research,  we  have  decided  to  expend  minimal  effort  on 
explaining  the  complexities  of  the  current  network  and  data  base  architectural 
technologies  that  exist  and  that  are  currently  being  re-engineered  through  the  Navy’s 
personnel  and  manpower  modernization  effort. 

b.  Navy  Personnel  Command  (NPC)  Project  Scope 

The  Navy  Personnel  Command’s  focus  and  scope  states  that  the  re¬ 
engineering  effort  must  design  a  visual  display  with  the  following  characteristics: 
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•  Compatible  with  the  Electronic  Military  Personnel  Records  Management 
System  (EMPRS)  or  the  technical  refreshed  version  of  EMPRS  currently  under 
development 

•  Compliant  within  the  NMCI  network  standards  and  operate  successfully  within 
enclave 

•  Should  have  compatibility  with  PeopleSoft  HRMS  applications 

•  Operational  within  Wood  Hall  Bldg.  769,  Navy  Personnel  Command 
(NAVPERSCOM) 

•  Will  provide  executable  design  for  selection  boards 

•  Will  not  alter  current  processes  or  require  changes  to  U.S.  Code  title  10 
promotion  law,  or  career  milestones 

•  Will  not  change  manpower  requirements  for  operations  and  support 

•  Increases  the  relative  effectiveness  of  visual  displays 

•  Provides  for  a  graphical  representation  of  OSR/ESR  (Officer/Enlisted 
Summary  Records)  data  reflecting  perfonnance  history,  awards,  and  promotion 
progression  set  against  a  career  timeline  or  history  of  assignments. 

•  Integral  into  the  project  portfolio  of  Sea  Warrior 

•  Successfully  extracts  data  for  the  Five  Vector  Model  (5VM)  from  the  Navy 
Personnel  Database  (NPDB)  and  the  Navy  Standard  Integrated  Personnel 
System  (NSIPS). 

2.  End  State 

The  primary  practical  focus  of  this  research  is  to  make  recommendations  in  order 
to  enhance  the  promotion  board  members’  comprehension  of  the  data  in  a  way  that  makes 
the  current  process  more  effective,  efficient,  and  objective.  This  is  an  important  factor 
that  may  contribute  to  immediate  short-term  results,  such  as  shortening  the  selection 
board  process  while  increasing  its  efficiency.  Long  term  effects  could  include  an  increase 
in  retention  rates  due  to  the  enhancement  of  personnel  satisfaction  attributed  to  a  more 
effective  detennination  of  the  future  promotion  or  assignment  of  Navy  personnel.  The 
overall  vision  is  for  a  two-stage  implementation  with  the  final  stage  using  the  enhanced 
displays  for  the  projection  of  the  Five  Vector  Model  for  promotion  purposes  in  the  future. 

The  concept  of  the  Five  Vector  Model  is  to  show  Naval  personnel  their  career 
path,  identify  the  requirements  they  need  to  complete,  and  to  qualify  them  for  future 
assignments.  The  model  aligns  professional  development  requirements  with  certifications 
and  qualifications  (military  and  civilian)  that  are  recognized  as  the  industry  standard. 
Once  the  Fleet  has  identified  the  requirements,  subject  matter  experts  (SMEs)  in  each 
field  plot  them  on  the  vector  model  where  an  individual  should  ideally  meet  those 
milestones  in  their  career.  The  model  also  considers  personal  development,  leadership, 
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management,  and  individual  performance.  (Ferron).  The  ultimate  end  state  and  radical  re¬ 
design  would  then  be  to  use  the  first  phase  re-design  of  the  visual  displays  to  show  the 
Five  Vector  Model  as  a  way  of  displaying  the  qualifications  and  individual  performance 
with  regard  to  promotion. 

The  immediate  benefit  of  this  study,  however,  will  be  an  improved  process  for 
effectively  and  efficiently  selecting  and  promoting  Naval  officers  by  improving  the  visual 
representation  of  perfonnance  information  for  the  selection  boards  while  minimizing 
selection  board  member  training,  reducing  record  preparation  time,  and  reducing  the 
mental  “data  to  information”  processing  time. 

B.  BACKGROUND 

The  Performance  Summary  Record  (PSR),  Officer  Summary  Record  (OSR)  and 
Enlisted  Summary  Record  (ESR)  are  used  by  selection  boards,  detailers,  and  other  career 
planners  to  detennine  the  future  promotion,  advancement  and  assignment  of  Navy 
personnel.  During  the  selection  board  process,  two  or  three  different  forms  are  displayed 
across  three  to  five  screens  in  a  small  theater-like  setting  referred  to  as  the  “tank.”  These 
forms  are  displayed  for  two  to  three  minutes  as  the  individual’s  record  is  reviewed  by  the 
board  membership.  It  is  essential  that  the  board  members  quickly  review  and  understand 
the  information  presented  on  the  summary  form.  The  summary  forms  are  currently 
designed  to  capture  a  combination  of  basic  personnel  infonnation  including  designator  or 
rating,  year  group,  education  history,  and  special  qualifications.  The  form  also  includes  a 
listing  of  duty  assignments  and  a  numeric  summary  of  performance  marks  that  the 
individual  under  consideration  has  received  throughout  his  or  her  career.  These  forms  are 
annotated  by  selection  board  members  and  used  to  brief  other  selection  board  members 
on  the  individual’s  career.  The  personnel  data  displayed  on  the  forms  are  predetermined 
and  there  is  no  ability  to  tailor  the  form  or  the  information  on  the  form  for  specific  types 
of  boards.  When  special  career  infonnation  is  required  for  a  particular  type  of  board,  that 
information  must  be  manually  entered  on  the  form  by  selection  board  administrative 
staff.  The  information  on  the  forms  is  extracted  from  several  legacy  data  systems  using 
outdated  information  retrieval  processes. 


4 


1.  Current  Process 

The  current  method  of  displaying  a  candidate’s  personnel  record  for  promotion  or 
selection  decisions  relies  primarily  on  human  cognition  to  process  tabular  O/ESR  and 
PSR  data  into  the  required  information  used  for  comparison  and  decision-making.  Board 
members  use  Officer/Enlisted  (O/E)  Summary  Record  Forms  and  Performance  Summary 
Records  to  gain  insight  into  the  candidate’s  personnel  data,  promotion/advancement 
history,  educational  background,  and  job  perfonnance  in  specific  billets  in  order  to 
evaluate  promotion  or  selection  potential.  Applicable  portions  of  the  candidate’s  record 
on  microfiche  are  used  to  add  more  detail  to  the  performance,  award,  education,  and 
special  qualifications  entries  made  on  the  summary  records.  A  system  to  provide  the 
candidate’s  data,  called  the  Electronic  Military  Personnel  Records  System,  has  been 
implemented  and  is  explained  in  further  detail  in  chapter  II. 

2.  Need  for  Improvement 

This  current  data-to-infonnation  conversion  process  relies  entirely  on  notes, 
symbols,  and  highlighted  annotations  on  the  digital  summary  record  forms  to  convey  the 
candidate’s  potential  to  other  board  members  for  promotion  or  selection  decisions  in  a 
short  three  minute  or  less  presentation.  The  fonnat  in  which  to  view  this  information 
currently  is  limited  and  burdensome  because  it  is  made  up  of  primarily  quantitative  data 
dispersed  widely  across  five  different  display  screens,  thus  forcing  board  members  to 
search  for  and  extract  relative  pieces  of  data  and  then  mentally  process  and  integrate  the 
data  into  useful  information.  This  restricted  form  of  information  presentation  makes  it 
difficult  to  compare  candidates  at  a  minimum,  and  may  conceivably  even  lead  to 
distortions  in  the  evaluation  process. 

C.  RESEARCH  QUESTIONS 

This  thesis  will  attempt  to  answer  the  principle  research  question  of  whether  data- 
information  visualization  best  practices  can  be  embedded  in  information  technology  in 
order  to  improve  the  overall  effectiveness  and  objectiveness  of  the  selection  board 
process.  Additionally,  will  integrating  web-based  technology  with  the  visualization 
techniques  reduce  the  time  required  to  prepare,  brief,  and  make  decisions  in  the 
promotion  process?  Can  web-based  data  visualization  technologies  serve  as  an  enabler  to 
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this  problem,  and  if  so,  what  are  the  relevant  criteria  for  deciding  which  technologies  are 
the  most  appropriate  in  this  regard?  What  is  an  appropriate  user  interface  design  for  this 
decision  problem? 

In  order  to  answer  these  questions,  our  research  will  explore  subordinate  and 
more  specific  questions  such  as  which  principles  of  graphical  practice  apply  to  this  type 
of  data  and  which  will  best  convert  the  data  into  meaningful  visual  infonnation  in  this 
decision  context?  Are  there  colors,  layering/separation,  or  multidimensional  schemes 
that  would  facilitate  infonnation  flow  without  inserting  bias  in  envisioning  a  candidate’s 
personal  and  performance  information?  We  will  also  introduce  the  Knowledge  Value 
Added  methodology  which  will  provide  us  with  a  performance  ratio  for  all  the  core 
processes  involved  in  order  to  determine  if  there  is  any  increase  in  effectiveness  between 
the  “as-is”  and  “to-be”  models. 

D.  METHODOLOGY 

1.  Proof  of  Concept  Context 

The  proof-of-concept  context  for  this  research  is  the  current  Naval  officer 
promotion/selection  process.  In  order  to  answer  the  principle  research  question,  the 
authors  have  developed  a  proof-of-concept  prototype  visualization  solution  through  the 
use  of  Commercial  off  the  Shelf  (COTS)  software  and  technology  in  order  to  demonstrate 
the  visualization  concepts  and  capabilities  currently  available.  This  prototype  uses  an 
artificial  database  and  simulates  the  current  system’s  input  and  outputs.  Using  three-tier 
architecture,  the  prototype  extracts  the  data  from  the  database  and  presents  an  improved 
visual  display  for  the  board  to  interpret  and  compare.  In  order  to  achieve  the  research 
goals,  we  have  completed  the  following  analysis  and  prototype  development: 

•  An  “as-is”  analysis  with  a  Return  on  Knowledge  (ROK)  spreadsheet  and 
graphical  representation. 

•  A  “to-be”  incremental  model  spreadsheet. 

•  A  fully  functional  web  site  that  utilizes  a  notional  relational  database. 

•  Prototype  web-based  charts  and  graphs  using  different  colors,  along  with 
layering  and  separation  multidimensional  schemes  that  would  facilitate 
information  flow. 
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2.  Literature  Research 

In  order  to  gain  extensive  insight  into  the  most  effective  and  objective  methods  of 
displaying  quantitative  information,  the  authors  have  conducted  an  in  depth  literature 
review  of  the  most  current  visualization  techniques  for  multi-criteria  decision-making 
models  in  conjunction  with  well  established  principles  for  the  display  of  visual 
information.  Much  of  this  theory  is  based  off  of  Edward  Tufte’s  classical  works  on  the 
visualization  of  data. 

Additionally  we  have  conducted  a  detailed  study  of  the  Knowledge  Value  Added 
methodology  and  have  primarily  focused  our  area  of  research  on  works  of  KVA’s 
foremost  authority  and  co-originator,  Dr.  Thomas  Housel. 

E.  ORGANIZATION  OF  THESIS 

This  thesis  is  organized  into  five  major  sections  or  chapters.  Chapter  I  is  the 
overview  of  our  research.  Chapter  II  is  primarily  devoted  to  explaining  the  promotion 
board  process  in  detail.  This  chapter  is  mainly  for  the  reader  with  little  or  no  background 
in  the  Naval  Promotion  Board  process.  Chapter  III  reviews  the  theory  and  literature  used 
in  the  research.  Chapter  IV  clarifies  the  details  of  the  Knowledge  Value  Added 
methodology  used  to  compare  the  current  and  future  system.  Chapter  V  will  introduce 
our  working  prototype.  Finally,  Chapter  VI  will  follow  up  with  a  conclusion,  some 
recommendations,  and  possible  areas  for  further  research. 
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II.  PROMOTION  BOARD  PROCESS 


A.  OVERVIEW 

This  chapter  will  help  the  reader  to  understand  the  promotion  processes  and 
procedures  that  are  detailed  in  the  remainder  of  this  document,  particularly  the  literature 
review  in  chapter  III.  It  is  particularly  important  for  the  reader  to  have  a  solid 
understanding  of  the  process  before  we  explain  our  proof  of  concept  methodology  in  the 
later  chapters.  We  briefly  outline  the  promotion  board  process  as  it  exists  today  and  cover 
the  basics  of  the  process.  With  this  chapter,  we  intend  to  inform  the  reader  who  has  not 
had  any  prior  experience  with,  or  knowledge  of,  the  Navy’s  promotion  board  process. 

There  are  two  types  of  boards:  Statutory  and  Administrative.  Statutory  boards 
include  promotion  (including  special  and  spot  promotion  boards),  Selective  Early 
Retirement  Boards  (SERB),  and  various  continuation  boards. 

All  other  boards  are  administrative.  Statutory  boards  are  governed  by  law, 
primarily  Title  10  of  the  U.S.  Code.  Administrative  boards  are  governed  by  instruction  or 
policy.  Statutory  boards  are  convened  by  the  Secretary  of  the  Navy  (SECNAV)  and  the 
results  are  approved  by  the  President,  Secretary  of  Defense  (SECDEF),  and  SECNAV. 
Administrative  boards  are  convened  by  the  Chief  of  Naval  Personnel  (CNP)  or  the 
Commander,  Navy  Personnel  Command  (CNPC)  and  the  results  are  approved  by 
CNP/CNPC. 

The  precept  is  a  document  that  is  signed  by  the  convening  authority  and  directed 
to  the  president  of  the  board,  giving  general  and  specific  guidance  to  the  board  regarding 
the  criteria  upon  which  their  selections  should  be  based.  The  precept  provides  several 
important  factors  for  the  board  to  consider,  such  as  board  membership,  promotion 
opportunity-stated  percentage  (i.e.  95  percent,  80  percent,  etc),  and  any  specific  guidance 
such  as  the  special  needs  of  the  Navy  at  that  time.  The  precept  is  the  only  guidance  for 
selection  provided  to  a  board.  Using  the  precept  as  guidance,  the  only  other  sources  of 
information  about  an  officer  allowed  for  consideration  by  a  statutory  board  are  the 
following: 
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•  Personnel  Record  Microfiche  -  contains  an  officer’s  fitness  reports 
(FITREPS),  photograph,  personal  awards  and  other  matters  of  official  record. 

•  Officer  Summary  Record  (OSR)/Performance  Summary  Record  (PSR)  -  an 
officer’s  career  resume  containing  a  summary  of  his  or  her  microfiche. 

•  Any  correspondence  the  officer  submits  to  the  board  about  their  record. 

No  information  other  than  what  is  listed  above  is  allowed  to  be  discussed  or  presented 
before  a  board. 

In  an  effort  to  narrow  the  scope  of  this  project,  we  decided  to  focus  our  effort  and 
base  our  proof  of  concept  solely  upon  the  officer  statutory  board  process.  In  particular, 
we  focused  on  the  promotion  boards  because  they  are  the  most  common  by  far  and  carry 
the  largest  impact. 

The  mission  of  any  board  is  to  select  those  officers  that  are  “best  qualified”  based 
on  performance.  Additional  guidance  in  the  precept  addresses  specific  guidance  for  the 
board  to  consider  in  detennining  the  best-qualified  officers.  The  rest  of  the  chapter  will 
illustrate  the  typical  promotion  board  process  and  will  be  broken  down  into  three 
sections: 

(1)  Pre-board 

(2)  Record  Review 

(3)  Selection. 

1.  Pre-Board  Phase 

All  boards  take  place  at  the  Navy  Personnel  Command  (NPC)  in  Millington, 
Tennessee.  The  initial  groundwork  for  a  promotion  board  begins  about  four  months 
before  its  convening  date  at  NPC  when  the  initial  list  of  eligible  officers  is  compiled  and 
modified  as  required.  The  eligible  list  is  continually  synchronized  with  an  official 
automated  database  to  ensure  consideration  of  all  candidates  and  the  master  file  is 
queried  six  weeks  prior  to  the  board  convening  for  FITREP  continuity.  Messages  are  then 
sent  out  for  any  missing  FITREPs.  The  images  that  are  reviewed  by  selection  boards,  and 
the  data  that  is  presented  on  the  OSR/PSR,  come  from  two  different  sources  and  there  are 
often  errors  and  inconsistencies  between  the  two.  Researching  and  correcting  those  errors 
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and  inconsistencies  is  performed  by  the  assistant  recorders  who  arrive  and  begin  their 
work  the  week  before  the  board  convenes.  This  process  is  called  “prep  week”. 

Board  members  arrive  at  NPC  on  the  date  authorized  in  the  precept  and  convene 
the  board.  Board  membership  is  secret  until  the  board  convenes  and  members  are 
specifically  directed  not  to  visit  detailers  prior  to  or  during  board  deliberations.  The 
process  is  designed  to  ensure  a  level  playing  field  for  all  involved.  Above  all,  board 
membership  is  carefully  balanced  to  represent  the  demographics  of  all  the  candidates  in 
the  selection  zones.  The  precept  is  discussed  and  the  board  begins  work  on  the  individual 
record  review. 

2.  Records  Review  Phase 

All  of  the  eligible  officers’  data  or  records  are  divided  up  and  impartially 
dispersed  among  the  voting  members  of  the  board  for  review.  Promotion  board  members 
review  a  digitized  form  of  the  official  record  for  each  eligible  officer.  Within  each 
eligible  officer’s  digitized  record,  the  board  members  review  the  officer’s  photograph, 
FITREPs,  OSR/PSR,  and  personal  awards  starting  from  the  Navy  Achievement  Medal 
and  higher.  Education,  promotion  history,  and  board  certification  are  some  of  the  other 
areas  of  interest  being  carefully  looked  at  during  this  phase.  Additionally,  the  board 
members  are  provided  and  privileged  information  (formerly  Fiche  5)  on  all  eligible 
officers,  if  applicable.  This  information  may  be  medical  or  punitive,  such  as  medical 
board  infonnation  or  non-judicial  punishment  awarded  at  Article  15  proceedings  and 
courts  martial  results.  Most  officers,  however,  do  not  have  any  privileged  information  in 
their  record. 

The  board  members  are  provided  the  OSR  and  PSR  which  are  administrative  tools 
used  to  track  individual  perfonnance.  Since  1995,  the  PSR  has  replaced  the  OSR  as  the 
sole  perfonnance-tracking  tool.  The  PSR  provides  a  quick  snapshot  of  the  eligible 
officer’s  education,  duty  assignments,  awards,  and  FITREP  marks,  while  excluding  the 
narrative  section  of  the  FITREP.  An  additional  source  of  information  reviewed  by  the 
board  is  correspondence  sent  by  the  eligible  officer  in  the  form  of  a  letter  directly  to  the 
president  of  the  board. 
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The  Electronic  Military  Personnel  Records  System  (EMPRS)  is  the  principal 
information  tool  used  by  the  board  members  to  review  a  candidate’s  record.  EMPRS  is  an 
image-based  information  management  tool  that  is  used  for  processing,  storing,  and 
distributing  both  digital  images  and  ASCII  data.  The  EMPRS  encompasses  three 
distinctive  modules  which  are  the  Primary  Input/Output  system  (PRIO),  the  Fitness 
Report/Evaluation  system  (FITREV)  and  Selection  Board  Module  (SELBD).  These 
modules  work  together  to  create  a  system  that  allows  operators  to  manage  member 
records  from  the  desktop  and  for  selection  board  members  to  electronically  evaluate 
personnel  for  promotion. 

Specifically,  the  board  members  will  sit  at  an  EMPRS  terminal  and  electronically 
review  all  of  the  pertinent  data  previously  compiled  and  downloaded  on  his  or  her 
assigned  candidates.  The  board  members  have  the  ability  to  not  only  review  but  also 
highlight,  mark  up,  and  annotate  the  PSR  from  the  EMPRS  terminal  by  means  of  a 
handwriting  tool  to  draw  lines,  circles,  and  various  other  pre-configured  auto  shape 
functions  that  allow  the  reviewer/briefer  the  ability  to  highlight  and  specifically  point  out 
to  other  board  members  pertinent  pieces  of  information  from  that  record  (Figure  1). 


Figure  1.  OSR/PSR  Cover  Page  displayed  on  EMPRS.  (From  BUPERS  Presentation) 
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The  board  carefully  considers,  without  prejudice  or  partiality,  the  record  of  every 
eligible  officer.  The  officers  selected  are  those  whom  a  majority  of  the  members  of  the 
board  consider  best  qualified  for  promotion,  giving  due  consideration  to  the  needs  of  the 
Navy  for  officers  with  particular  skills.  In  addition  to  the  standard  of  best  qualified,  all 
officers  recommended  for  promotion  must  be  fully  qualified;  that  is,  each  officer  must  be 
capable  of  perfonning  the  duties  of  the  next  higher  grade. 

3.  Selection  Phase 

After  the  records  review  phase,  the  board  moves  on  to  the  next  step  -  the  selection 
phase.  In  this  phase,  the  board  members  move  into  a  room  called  “The  Tank”  which  is  a 
private,  theatre-like  room  where  all  the  members  discuss  and  vote  on  candidates.  The 
annotated  OSR/PSRs  are  projected  onto  five  large  screens  in  the  tank,  (Figure  2)  and 
each  record  is  briefed  by  the  board  member  who  reviewed  the  candidate’s  record. 

After  the  briefing  officer  has  discussed  the  candidate  and  all  questions  have  been 
asked  and  answered,  each  member  uses  a  “secret  ballot”  computer  keypad  located  on  the 
ann  of  their  seats  to  cast  a  confidence  level  vote  for  the  selection  of  the  candidate.  Each 
member  can  vote  either  100  percent  (the  member  is  100  percent  sure  the  candidate  should 
be  selected),  75  percent,  50  percent,  25  percent  or  0  percent  (the  candidate  should  not  be 
selected  ).  After  all  of  the  votes  are  cast,  the  computer  in  the  tank  combines  them  into  an 
overall  confidence  rating  which  is  then  displayed  as  an  average  percentage  on  a  monitor 
for  all  the  board  members  to  see. 

The  confidence  rating  of  each  candidate  is  recorded  and  then  ranked  after  all  the 
records  have  been  reviewed.  A  board  member  (usually  the  senior  member  or  president  of 
the  board)  will  propose  an  option  for  selection  to  the  entire  board.  The  board  selects  a 
number  of  the  records  from  the  top  scorers  to  be  “tentatively  selected”  or  what  we’ll  call 
the  TS  category.  The  board  will  either  vote  on  the  motion  or  members  will  offer  counter¬ 
proposals.  Whichever  proposal  is  accepted,  it  is  accepted  by  a  majority  vote  of  all  the 
members.  This  same  scenario  is  repeated  when  the  board  attempts  to  determine  which 
number  of  the  bottom  scoring  candidates  should  be  “dropped  from  further  consideration” 
and  those  candidates  will  be  placed  in  the  DFC  category.  All  of  the  candidates  between 
the  TS  and  the  DFC  categories  fall  into  what  is  called  the  “crunch”  category  or  zone. 
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Each  of  the  candidates  that  fall  into  the  “crunch”  category  is  reviewed  by  a 
different  board  member  than  the  one  who  originally  briefed  their  record.  Each  candidate 
receives  another  review  and  brief  in  the  tank  and  the  process  starts  over  again  and  is 
repeated  until  eventually  the  entire  crunch  list  is  diminished  and  all  candidates  have 
either  been  selected  or  dropped  from  further  consideration.  Several  “tank”  sessions  are 
usually  required  before  the  board  selects  the  final  number  of  candidates  they  believe  are 
best  qualified  for  promotion.  Finally,  the  board  then  completes  its  deliberations  and  votes 
to  confirm  the  tentative  selections. 

The  following  sections  will  detail  exactly  what  the  board  is  looking  at  when 
voting  in  the  “tank.”  This  will  give  the  reader  a  better  understanding  of  how  and  why  we 
designed  our  proof  of  concept  model. 


Figure  2.  OSR/PSR  info  displayed  in  the  “tank”.  (From  BUPERS  Presentation) 

B.  PERFORMANCE  SUMMARY  RECORD  (PSR) 

The  following  section  will  discuss  how  the  boards  use  the  Officer  Summary 
Record  (OSR)/Performance  Summary  Record  (PSR)  to  make  their  selections.  The 
OSR/PSR  is  an  administrative  tool  that  summarizes  an  officer’s  professional  and 
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performance  history  and  serves  as  the  boards  “snap  shot”  of  the  candidate’s  career.  The 
board  member  who  reviewed  and  annotated  the  candidates  data  on  the  EMPRS  will  brief 
that  record  in  the  “tank”  and  is  the  annotated  OSR/PSRs  that  he  or  she  is  summarizing 
to  the  other  board  members.  The  OSR  consists  of  the  cover  page  (Figure  3)  and  FITREP 
detail  pages  that  document  the  old  FITREP  system  reports.  The  PSR  consists  of  the  new 
system  reports.  Eet’s  take  a  look  at  a  sample  OSR/PSR  and  look  at  this  notional  record  as 
a  selection  board  member  would  do  during  a  board.  In  an  effort  to  keep  this  concise  and 
to  limit  the  scope  of  our  proof  of  concept  model,  we  will  be  reviewing  only  the  new 
FITREP  evaluation  page  (PSR)  and  not  the  old  FITREP  evaluation  page  (page  2  of  the 
OSR  proper). 

1.  OSR/PSR  Cover  Page 

When  a  board  reviews  the  OSR/PSR  data  they  start  with  the  cover  page.  The 
OSR  cover  page  consists  mainly  of  administrative  data  such  as  the  candidates  name, 
social  security  number,  and  career  history.  The  comments  in  our  example  (Figure  3)  are 
typed  on  the  OSR/PSR  image  shown  on  the  briefer’s  EMPRS  screen  and  the  circles  and 
lines  are  drawn  using  computer  graphics.  The  briefer  (the  board  member  who  reviewed 
the  candidate’s  record)  made  these  remarks  to  highlight  important  points  in  the 
candidate’s  career  to  the  other  board  members  when  they  are  voting  in  the  tank. 


Figure  3.  OSR/PSR  Cover  Page  EMPRS  view.  (From  BUPERS  Presentation) 
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Apart  from  the  standard  information  (name,  SSN,  date  of  birth,  etc.)  at  the  top, 
notice  that  the  briefer  has  annotated  several  items  that  he  deemed  important  to  point  out 
to  other  board  members.  For  example,  we  can  see  that  the  briefer  points  out  this 
candidate  as  an  E-2  C.O.  and  was  “ALWAYS  ABOVE  THE  PACK,  COMMUNITY 
LEADER,  RECOMMENDED  FOR  FRS  C.O.  BY  BERNIE  SMITH.”  We  can  also  see 
that  another  briefer  (different  color  marks)  noted  and  circled  comments  such  as  CMD, 
E2C  EXPERT,  POST  GRAD,  AND  1%,  all  of  which  could  help  the  other  board 
members  with  their  decisions  in  the  tank. 

2.  PSR 

The  following  is  a  brief  description  of  how  the  “new  system”  FITREPs  are 
displayed  in  the  PSR.  The  new  FITREP  system  took  affect  in  August  1995  and 
completely  changed  the  FITREP  and  the  look  of  the  PSR.  From  Figure  4  we  can  see  from 
looking  at  the  top  line  and  reading  across  from  left  to  right,  that  this  individual’s  present 
grade  (PG)  is  05.  We  can  also  see  that  he  was  the  Commanding  Officer  of  VAW-126  and 
that  this  particular  reporting  period  (the  top  line)  was  from  090195  to  041296.  The  report 
covers  a  7  month  period  and  his  reporting  senior  who  wrote  this  report  is  T.E  Zelibor 
whose  present  grade  is  06  and  his  title  at  the  time  of  the  report  was  Commander  (COM). 

The  remainder  of  the  columns  across  the  page  contain  the  individual  numbers  and 
grade  averages  that  make  up  the  meat  of  the  report.  For  trait  grades,  this  candidate 
received  two  “4.0”  grades  and  five  “5.0”  grades.  His  individual  average  was  4.71  and  the 
reporting  senior’s  (RS)  trait  grade  average  for  this  summary  group  was  4.60.  This 
candidate  was  in  a  summary  group  (competitive  category)  consisting  of  6  individuals 
(add  3,  1,  2  reflected  in  the  summary  line  of  the  promotion  recommendation  column). 
The  total  number  of  Commanders  (05  rank/grade)  for  which  Capt  Zelibor  was  the 
reporting  senior,  regardless  of  designator  (Active,  and  Reserve,  Line  and  Staff),  was  7 
(Top  number  in  CUM  column).  Capt  Zelibor’s  cumulative  average  for  the  7  Commanders 
he  had  evaluated  up  to  the  end  date  of  this  report,  (including  the  6  in  this  summary  group) 
was  4.61  (Bottom  number  in  the  CUM  column).  This  individual  was  given  a  promotion 
recommendation  of  “Must  Promote”  (X  under  “MP”)  along  with  one  other  in  the 
summary  group.  Three  individuals  in  the  summary  group  were  given  “Promotable” 
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recommendations  (3  under  the  “PR”)  and  two  were  given  “Early  Promote”  (2  under 
“EP”)  promotion  recommendations.  The  report  was  a  regular  (“RG”)  report,  which 
maintained  his  record  continuity  (concurrent  and  Operational  Commander  Reports  do  not 
count  for  continuity). 

From  Figure  4  we  can  see  that  the  two  briefers  who  reviewed  this  record  on  the 
EMPRS  noted  that  this  individual  was  “ALWAYS  ABOVE  THE  PACK  AS  C.O.  AND 
ALWAYS  RANKED.”  Also  annotated  was  the  fact  that  he  was  a  “CLF  SILVER 
ANCHOR  WINNER”  during  the  top  reporting  period.  The  heavy  line  drawn  across  the 
middle  of  the  report  is  just  separating  two  different  Commands. 
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Figure  4.  PSR  with  briefers’  annotated  marks.  (From  BUPERS  Presentation) 


3.  Summary 

In  this  chapter  we  have  reviewed  the  Promotion  Board  process  with  the  intent  to 

explain  the  basics  of  the  procedures  to  the  reader  who  has  had  little  or  no  prior 

knowledge  of,  or  experience  with,  the  Navy’s  promotion  process.  We  believe  this  will 

provide  a  solid  foundation  from  which  the  reader  may  better  derive  a  thorough 

understanding  of  our  methodology  and  proof  of  concept  model.  From  the  last  section  it 

should  be  fairly  clear  to  the  reader  why  we  believe  there  could  be  a  more  effective  way  in 
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which  to  display  the  OSR/PSR  information  to  the  board  members  in  the  tank.  Much  of 
the  pertinent  information  is  quantitative,  convoluted,  and  widely  dispersed  across  the  five 
screens  in  the  tank.  The  following  chapters  will  explain  the  theory  behind  our  proposal, 
an  economic  comparison  of  the  current  system  to  what  we  are  proposing,  and  a  look  at 
the  proof  of  concept  prototype  followed  by  the  conclusion  and  recommendations. 
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III.  LITERATURE  REVIEW 


A.  INTRODUCTION 

The  current  method  for  displaying  a  candidate’s  personnel  record  for  promotion 
or  selection  decisions  relies  heavily  on  human  cognition  to  process  tabular  OSR/PSR  data 
into  the  required  information  used  for  comparison  and  decision  making.  The  majority  of 
the  information  is  numerical  and  is  often  spread  out  across  different  places  among  the 
five  large  screens  within  the  tank.  Our  challenge  is  how  can  we  arrange  the  data 
graphically  or  visually  so  that  it  is  aesthetic  to  the  viewing  board  members  in  order  to 
allow  them  to  more  easily  correlate  all  of  the  data  quickly  and  objectively?  Can  we  make 
it  web-based  so  that  some  of  the  extremely  time  consuming  manual  processes  such  as 
scanning  documents  onto  the  system  or  having  to  separately  download  data  into  the  tank, 
be  eliminated  completely?  How  do  we  effectively  measure  this  to  see  if  it’s  worth  the 
effort  to  redesign?  In  other  words  how  do  we  go  about  measuring  the  change  in 
performance  from  the  “as  is”  baseline  process  in  use  today  against  our  proposed  “to  be” 
model  of  tomorrow? 

This  chapter  will  review  the  literature  we  used  in  order  to  answer  the  previous 
questions.  Section  B  discusses  the  Knowledge  Value  Added  (KVA)  methodology  that 
was  developed  by  Dr.  Thomas  J.  Housel  and  Dr.  Valery  Kanevsky.  KVA  is  essentially  a 
performance  metric  that  provides  a  means  for  objective  measurement  of  the  relationship 
between  knowledge  and  value  in  organizational  processes.  Section  C  will  detail  the  best 
practices  for  effectively  displaying  quantitative  information  graphically  as  seen  by  one  of 
the  renowned  authorities  in  that  field,  Edward  R.  Tufte. 

B.  KNOWLEDGE  VALUE  ADDED  (KVA) 

Omar  A.  El  Sawy,  in  his  book  Redesigning  Enterprise  Processes  for  e-Business, 
explains  that  one  of  the  central  tenets  of  the  Knowledge  Value-Added  (KVA) 
methodology  is  that: 
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When  a  process  is  redesigned  to  be  more  knowledge-creating,  it  is  often 
hard  to  measure  the  added  value  to  the  customer  of  the  process.  One  of  the 
characteristics  of  such  a  redesign  principle  is  that  it  causes  the  business 
process  to  learn  more  from  the  participants  it  interacts  with  each  time  there 
is  an  interaction.  Thus  the  value  of  the  process  increases  over  time  as 
intellectual  capital  through  repeated  execution,  even  though  initially  there 
is  no  immediate  added  value  for  the  customer.  It  is  difficult  at  the  design 
stage  to  measure  what  the  return  on  redesigning  the  process  might  be.  The 
quicker  that  the  new  knowledge  in  the  process  can  be  translated  into  value 
that  the  customer  is  willing  to  pay  for  (whether  through  this  business 
process  or  a  related  one),  the  more  effective  the  business  process  design  is 
(El  Sawy,  2001). 

How  can  we  adequately  measure  the  effectiveness  of  what  we’re  proposing?  It 
would  be  very  difficult,  to  measure  whether  or  not  the  board  members  were  making 
better  decisions  with  the  redesigned  process.  Although  the  redesigned  process  would 
result  in  shorter  board  times  and  thus  resulting  in  less  costs,  we  would  be  remiss  if  we 
were  to  base  the  effectiveness  solely  on  time  savings.  Much  of  the  change  that  we  need  to 
measure  is  intrinsic  and  may  not  be  immediately  apparent.  For  example,  better  decisions 
made  in  the  promotion  and  assignments  processes  of  today  may  lead  to  an  improved 
quality  of  life  and  higher  retention  rates  for  tomorrow. 

1.  Knowledge  Value  Added  Theory 

The  method  we  chose  to  measure  the  benefits  of  greater  efficiency  in  the  decision 
process  is  the  Knowledge  Value  Added  (KVA)  methodology.  This  methodology  is 
beneficial  to  non-profit  organizations  because  it  presupposes  knowledge  as  a  surrogate 
for  value  and  therefore  can  be  used  independent  of  profit  generation.  Because  of  this  and 
the  fact  that  cutting  costs  and  reducing  head  count  were  not  viable  options  for  our  proof 
of  concept,  KVA  was  chosen  because  it  enables  managers  to  measure  the  performance  of 
corporate  or  community  knowledge  assets  whether  the  knowledge  is  deployed  in 
information  technology  (IT)  or  resident  within  employees’  heads. 

KVA  analysis  produces  a  Return  on  Knowledge  (ROK)  ratio  to  estimate  the  value 
added  by  given  knowledge  assets  regardless  of  where  they  are  located  within  the 
organization.  The  essence  of  KVA  is  that  knowledge  utilized  in  organizations’  core 
processes  is  translated  into  numerical  form.  This  translation  allows  allocation  of  revenue 
in  proportion  to  the  value  added  by  the  knowledge  as  well  as  the  cost  to  use  that 
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knowledge.  Tracking  the  conversion  of  knowledge  into  value  while  measuring  its  bottom 
line  impacts  enables  managers  to  increase  the  productivity  of  these  critical  assets  (Housel 
and  Bell,  2001). 

KVA  provides  a  methodology  for  allocating  revenue  and  cost  to  an  organization’s 
core  processes  based  on  the  amount  of  change  each  produces.  KVA  standardizes  the 
output  of  all  processes  by  describing  the  output  in  tenns  of  the  units  of  knowledge 
required  to  produce  it.  Notably,  the  knowledge  required  to  make  these  changes  is  a 
convenient  way  to  describe  the  conversion  process.  Knowledge  is  the  know-how  required 
to  produce  process  outputs,  process  instructions,  or  in  our  case,  process  decisions.  This 
kind  of  knowledge  is  proportionate  to  the  time  it  takes  to  learn  it. 

2.  Knowledge  Value  Added  Method 

The  KVA  method  is  extremely  beneficial  to  organizations  because  it  is  simple 
enough  to  be  applied  in  seven  steps  yet  it  is  robust  enough  to  produce  a  desired  level  of 
granularity  should  the  organization  desire  a  more  comprehensive  view  of  their  processes. 
Housel  and  Bell  offer  three  different  ways  to  establish  the  value  of  knowledge  embedded 
in  the  organizations  core  processes  (Table  1). 


Stops 

Learning  time 

Process  description 

Binary  query  method 

1. 

Identify  core  process  and  its  subprccesses. 

2. 

Establish  common  units  to 
measure  learning  time. 

Describe  the  products  in  terms 
of  the  instructions  required  to 
reproduce  them  and  select  unit 
of  process  description. 

Create  a  set  of  binary  yes/no 
questions  such  that  all  possible 
outputs  are  represented  as  a 
sequence  of  yes/no  answers. 

3. 

Calculate  learning  time  to 
execute  each  subprocess. 

Calculate  number  of  process 
instructions  pertaining  to 
each  subprocess. 

Calculate  length  of  sequence  of 
yes/no  answers  for  each 
subprocess. 

4. 

Designate  sampling  time  period  long  enough  to  capture  a  representative  sample  of  the  core 
process's  final  product/service  output. 

5. 

Multiply  the  learning  time  for 
each  subprocess  by  the  num¬ 
ber  of  times  the  subprocess 
executes  during  sample  period. 

Multiply  the  number  of  process 
instructions  used  to  describe 
each  subprocess  by  the  number 
of  times  the  subprocess 
executes  during  sample  period 

Multiply  the  length  of  the  yes/no 
string  for  each  subprocess  by  the 
number  of  times  this  subprocess 
executes  during  sample  period. 

6. 

Allocate  revenue  to  subprocesses  in  proportion  to  the  quantities  generated  by  step  5  and  calculate 
costs  for  each  subprocess. 

7.  Calculate  ROK,  and  interpret  the  results. 


Table  1  Three  Approaches  to  KVA  (From  Housel  and  Bell) 
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Although  the  binary  query  method  is  usually  the  most  precise  method,  it  is  also 
the  most  time  intensive  and  is  usually  only  used  for  situations  requiring  a  high  degree  of 
accuracy  and  granularity.  The  method  we  chose  for  our  Proof  of  Concept  is  the  Learning 
Time  method.  This  method  allows  those  who  use  it  to  establish  rough-cut  estimates  of  the 
value  of  knowledge  within  processes.  It  can  be  accomplished  more  quickly  than  the 
binary  query  method  and  is  targeted  at  the  aggregate  level  of  analysis.  An  example  of  a 
high-level  aggregate  KVA  analysis  is  shown  in  Table  2. 
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Table  2  High-level  Aggregate  KVA  Analysis  (From  Housel  and  Bell) 


Table  2  shows  an  example  of  a  company’s  results  of  a  seven  step  high-level  KVA 
analysis.  Right  away  managers  are  able  to  determine  the  relative  perfonnance  of  the  core 
functional  areas  in  terms  of  ROK.  The  results  would  serve  as  a  starting  point  from  which 
a  more  detailed  KVA  analysis  could  be  done  to  identify  knowledge  embedded  in  sub¬ 
processes  and  to  help  managers  make  improved  decisions  on  how  to  increase  the 
company’s  profits. 

Learning  Time  (LT)  has  been  established  as  a  fast  and  appropriate  way  to 
measure  the  amount  of  knowledge  contained  in  any  given  process.  The  amount  of 
knowledge  embedded  in  a  process  corresponds  to  the  amount  of  time  necessary  for  an 
average  person  to  leam  how  to  complete  the  process  properly.  The  knowledge  can  also  be 
represented  by  the  process  instructions  required  to  produce  the  process  output  effectively. 
Learning  time  is  in  proportion  to  the  amount  of  knowledge  learned.  In  this  manner, 
learning  time  can  be  used  as  a  common-sense  indicator  of  the  amount  of  knowledge 
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embedded  within  a  given  process.  Usually  the  Subject  Matter  Experts  (SME)  for  a 
particular  process  can  provide  actual  realistic  estimates  of  the  learning  time  required  for  a 
given  process  based  on  formal  and  informal  training  times,  experience  on  the  job, 
employee  interviews,  training  manuals,  and  programs. 

3.  KVA  Example 

The  following  is  a  short  and  extremely  simple  example  of  KVA  and  how  it  works, 
taken  from  Housel  and  Bell’s  Measuring  and  Managing  Knowledge.  In  the  Widget 
Company,  there  is  one  person,  the  owner,  who  makes  and  sells  widgets.  This  person 
knows  all  there  is  to  know  in  order  to  make  and  sell  widgets  for  $1.  The  owner’s  sales- 
production  knowledge  can  be  used  as  a  surrogate  for  the  dollar  of  revenue  generated  by 
his  application  of  the  core  process  knowledge.  We  can  determine  how  long  it  takes  the 
widget  company  owner  to  transfer  all  the  necessary  sales  and  production  knowledge  to  a 
new  owner.  Further,  we  can  use  these  leaning  times  to  allocate  the  dollar  of  revenue 
between  the  sales  and  production  processes.  For  simplicity  sake  lets  say  that  it  takes  100 
hours  for  someone  to  learn  a  new  process  with  70  hours  spent  learning  how  to  make  the 
widget  and  30  hours  learning  how  to  sell  it.  This  would  indicate  that  70  percent  of  the 
knowledge  and  value  added  was  contained  on  the  production  process  and  30  percent  in 
the  sales  process.  It  would  follow  that  $.70  of  the  revenue  would  be  allocated  to 
production  knowledge  and  $.30  to  sales  knowledge. 

All  that  would  be  left  to  do  in  this  example  would  be  to  determine  how  much  it 
costs  to  use  the  sales  and  production  knowledge  and  then  we  would  have  a  ratio  of 
knowledge  value  added  to  knowledge  utilization  cost.  In  other  words,  we  can  measure 
return  on  knowledge  (ROK).  For  the  sake  of  argument,  we  will  assume  that  the  total  cost 
to  sell  and  produce  a  widget  was  $.50:  with  $.25  for  sales  and  $.25  for  production.  The 
basic  approach  here  is  to  find  out  how  much  it  costs  to  use  the  sales  and  production 
knowledge.  In  this  case,  the  cost  is  directly  tied  to  how  long  the  new  owner  spends 
performing  each  process.  As  it  turns  out,  in  this  case,  the  new  owner  spends  the  same 
amount  of  time  to  do  both  and,  therefore,  the  cost  to  use  the  knowledge  of  each  process  is 
the  same. 

Based  on  the  estimates  for  the  distribution  of  revenue  and  cost,  we  would  generate 

an  estimate  of  the  ROK.  We  would  conclude  that  the  production  process  is  a  more 
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productive  use  of  the  knowledge  asset  (ROK  =  ,7/.25=280%)  that  the  sales  process 
(ROK=.30/.25  =  120  percent). 

4.  KVA  Summary 

KVA  provides  the  organization  with  a  way  of  making  decisions  on  how  best  to 
shift  or  eliminate  knowledge  as  in  the  case  of  whether  or  not  to  automate  a  core  process. 
The  key  issue,  and  one  that  applies  to  our  case,  is  how  to  redeploy  knowledge  from 
people  and  procedures  or  work  rules  in  to  IT  so  that  it  can  be  executed  more  rapidly, 
therefore  more  often,  and  at  a  lower  cost.  If  the  ROK  of  a  given  process  is  not  improved 
with  the  implementation  of  the  automation,  then  steps  must  be  taken  to  improve  its 
functionality  and  perfonnance. 

The  same  view  may  be  extended  to  estimate  the  amount  of  knowledge  contained 
in  IT.  To  do  this,  the  organization  must  choose  IT  outputs  within  the  core  process  and 
then  ask  the  resident  SME  to  estimate  the  time  necessary  for  learning  how  to  generate  the 
same  outputs.  In  other  words,  he  or  she  should  estimate  how  long  it  would  take  to  do  the 
same  process  without  the  IT.  How  long  would  it  take  to  teach  the  average  person  to 
produce  the  same  output  manually  without  the  IT?  Most  likely,  it  would  take  much 
longer  for  the  average  person  to  produce  the  same  outputs  but  what  really  needs  to  be 
determined  is  how  long  it  would  take  that  person  to  learn  to  produce  the  outputs  (i.e.  the 
learning  time  estimate). 

In  this  way,  IT  is  just  another  layer  of  process  knowledge.  KVA  also  allows  the 
organization  to  detennine  that  important  question  of  what  is  its  return  on  IT?  KVA 
allows  the  organization  to  obtain  a  true  snapshot  of  the  return  it’s  getting  from  IT. 
Because  the  numerators  and  denominators  come  from  different  data  sources  (real 
revenues  and  costs)  managers  won’t  be  able  to  manipulate  cost  (i.e.  the  denominator)  to 
obtain  the  numerator  (e.g.  cost  savings  or  cost  avoidance)  so  that  a  false  return  would  be 
generated. 

C.  BEST  PRACTICES  FOR  VISUALIZATION  TECHNIQUES 

What  we  see  is  not  always  what  we  think  we  see.  We  must  ensure  that 

what  we  portray  is  not  misleading  or  misunderstood,  and  that  we  are 

conveying  the  intended  message  (Brown,  1995). 
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This  section  will  explore  the  current  best  practices  for  displaying  quantitative 
information  and  make  recommendations  on  how  to  most  effectively  and  objectively 
display  the  OSR/PSR  information.  One  of  the  greatest  benefits  of  data  visualization  is  the 
sheer  quantity  of  information  the  audience  can  rapidly  interpret  if  the  visual  data  is 
presented  well.  We  have  chosen  to  follow  the  theoretical  research  of  Edward  R.  Tufte 
because  of  his  renowned  expertise  in  this  field. 

This  chapter  outlines  Edward  Tufte’s  pioneering  work  on  the  use  of  visual 
graphics  to  display  quantitative  infonnation  and  it  mainly  consists  of  text  and  ideas  taken 
from  three  of  his  books  on  this  subject  matter.  We  have  included  a  few  diagrams  as 
examples  but  the  reader  should  examine  the  books  in  order  to  get  a  full  understanding 
and  appreciation  of  the  graphics  and  concepts  he  introduces. 

Modern  data  graphics  can  do  much  more  than  simply  substitute  for  small 
statistical  tables.  At  their  best,  graphics  are  instruments  for  reasoning 
about  quantitative  information.  Often  the  most  effective  way  to  describe, 
explore,  and  summarize  a  set  of  numbers  -  even  a  very  large  set  -  is  to 
look  at  pictures  of  those  numbers.  Furthermore,  of  all  methods  for 
analyzing  and  communicating  statistical  information,  well-designed  data 
graphics  are  usually  the  simplest  and  at  the  same  time  the  most  powerful 
(Tufte,  2001). 

1.  Graphical  Principles 

In  his  book  The  Visual  Display  of  Quantitative  Information,  Edward  R.  Tufte 
explains  that  excellence  in  statistical  graphics  consists  of  complex  ideas  that  are 
communicated  with  clarity,  precision,  and  efficiency.  He  further  notes  that  graphical 
displays  should: 

•  Show  the  data. 

•  Induce  the  viewer  to  think  about  the  substance  rather  than  about  methodology, 
graphic  design,  the  technology  of  the  graphic  production  or  something  else. 

•  Avoid  distorting  what  the  data  have  to  say. 

•  Present  many  numbers  in  a  small  space. 

•  Make  large  data  sets  coherent. 

•  Encourage  the  eye  to  compare  different  pieces  of  data. 

•  Reveal  the  data  at  several  levels  of  detail,  from  a  broad  overview  to  the  fine 
structure. 

•  Serve  a  reasonably  clear  purpose:  description,  exploration,  tabulation,  or 
decoration. 

•  Be  closely  integrated  with  the  statistical  and  verbal  descriptions  of  a  data  set. 
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In  order  to  gain  the  most  benefit  from  redesigning  the  promotion  board  displays 
so  that  the  data  is  clear,  easily  understood,  and  not  biased  in  any  manner,  we  adhered  to 
Tufte’s  principles  of  graphical  excellence.  These  principles  are  as  follows: 

•  Graphical  excellence  is  the  well-designed  presentation  of  interesting  data  -  a 
matter  of  substance,  of  statistics,  and  of  design. 

•  Graphical  excellence  consists  of  complex  ideas  communicated  with  clarity, 
precision,  and  efficiency. 

•  Graphical  excellence  is  that  which  gives  to  the  viewer  the  greatest  number  of 
ideas  in  the  shortest  time  with  the  least  ink  in  the  smallest  space. 

•  Graphical  excellence  is  nearly  always  multivariate. 

•  Graphical  excellence  requires  telling  the  truth  about  the  data. 

a.  Graphical  Integrity 

Tufte’s  works  are  appropriate  for  our  study  because  he  has  developed  a 
consistent  approach  to  the  display  of  graphics  which  enhances  its  dissemination, 
accuracy,  and  ease  of  comprehension.  Most  importantly,  he  stresses  that  you  cannot  gain 
graphical  excellence  without  telling  the  truth  about  the  data  which  is  paramount  in  our 
proposal  for  the  redesigned  promotion  displays. 

He  explains  that  the  problem  lies  in  presenting  large  amounts  of 
information  in  a  way  that  is  compact,  accurate,  adequate  of  the  purpose,  and  easy  to 
understand.  Specifically,  a  designer  must  show  cause  and  effect,  insure  that  the  proper 
comparisons  are  made,  and  achieve  the  valid  goals  that  are  desired.  A  major  problem 
with  presenting  the  visual  display  of  quantitative  information  is  the  misperception  of  that 
data  by  the  viewer.  Different  people  see  the  same  objects  somewhat  differently  and 
context-dependent  perceptions  change  with  experience  .  Another  factor  that  could  distort 
the  data  involves  miscommunication.  A  person’s  perception  of  an  object  can  change 
depending  on  what  others  have  already  said. 

In  order  to  reduce  these  effects,  the  designer  needs  to  strive  for  uniformity 
in  graphics  and  assurance  that  the  representation  accurately  depicts  the  target  numbers. 
Tufte  mentions  two  goals  that  if  followed,  may  alleviate  these  problems. 
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•  The  representation  of  numbers,  as  physically  measured  on  the  surface  of  the 
graphic  itself,  should  be  directly  proportional  to  the  numerical  quantities 
represented. 

•  Clear,  detailed,  and  thorough  labeling  should  be  use  to  defeat  graphical 
distortion  and  ambiguity.  Write  out  explanations  of  the  data  on  the  graphic 
itself.  Label  important  events  in  the  data. 

Tufte  theorizes  that  in  order  to  get  an  idea  of  how  much  a  graphic  is  in 
violation  of  the  first  principle,  one  could  measure  it  by  use  of  the  “Lie  Factor”  which  is 
the  size  of  effect  shown  in  the  graphic  divided  by  the  size  of  effect  in  data  [=(size  of 
graphic)/(size  of  data)].  The  Lie  Factor  is  used  to  exaggerate  differences  or  similarities.  If 
the  Lie  Factor  is  equal  to  one,  then  the  graphic  is  probably  doing  an  adequate  job  of 
accurately  representing  the  numbers.  If  the  Lie  Factor  is  greater  than  1.05  or  less  than  .95, 
then  substantial  distortion  is  possibly  involved.  As  an  example,  Tufte  uses  a  newspaper 
that  reported  that  the  U.S.  Congress  and  the  Department  of  Transportation  had  set  a  series 
of  fuel  economy  standards  for  automobile  manufactures,  beginning  with  18  miles  per 
gallon  in  1978  and  moving  incrementally  up  to  27.5  miles  per  gallon  by  1985,  an  increase 
of  53  percent.  However,  when  the  magnitude  of  the  change  from  1978  to  1985  is  shown 
in  the  graph  by  the  relative  lengths  of  the  two  lines,  it  reveals  a  difference  of  783  percent. 
Thus,  the  numerical  change  of  53  percent  is  presented  by  two  lines  that  changed  783 
percent,  yielding  a  Lie  Factor  of  14.8  percent,  well  beyond  the  bounds  of  acceptability. 

The  Lie  Factor  could  be  a  quick  and  easy  method  for  gaining  insight  into 
the  scale  of  the  graphics  used  to  depict  candidates’  PSR  infonnation  to  the  board.  For 
example,  suppose  a  candidate  had  a  trait  average  of  4.80  for  a  reporting  period  and  during 
the  subsequent  period,  he  had  a  report  average  4.60  for  a  negative  .20  difference  which  is 
a  4.16  percent  decrease  of  the  first  mark  of  4.80.  If  the  graph  depicting  that  difference 
changed  from  1.25  inches  to  1.00  inch,  we  calculate  a  decrease  of  a  20  percent  for  a  Lie 
Factor  of  (.20/.0416  =  4.81)  which,  in  accordance  with  Tufte’s  advice  is  substantially 
distorted  because  it  is  greater  than  the  1.05  upper  limit  that  he  recommends.  In  this  case 
you  would  most  likely  increase  the  scale  of  the  graph  (otherwise  the  difference  may  not 
be  visible  to  the  viewers)  which  would  have  the  effect  of  decreasing  the  difference  or 
variation  being  shown. 


27 


The  accurate  depiction  of  data  also  means  that  design  variation  should  not 
affect  or  distort  the  data  being  displayed.  In  other  words,  one  should  not  change  the  data 
in  order  to  make  the  graphic  look  nice.  The  compounding  of  design  variation  with  data 
variation  over  the  surface  of  a  graphic  leads  to  ambiguity  and  deception,  because  the  eye 
may  mix  up  changes  in  the  design  with  changes  in  the  data.  The  graphic  should  follow 
the  principle  of  showing  data  variation,  not  design  variation. 

Another  way  to  confuse  data  variation  with  design  variation  is  to  use  the 
space  to  show  one-dimensional  data.  A  common  mistake  is  to  use  areas  to  show 
magnitudes  and  falsely  varying  both  dimensions  simultaneously  in  response  to  changes  in 
one-dimensional  data.  Vast  ambiguities  arise  when  viewers  perceive  a  two-dimensional 
surface  and  then  convert  that  perception  into  a  non-dimensional  number.  The  difference 
in  physical  area  on  the  surface  of  a  graphic  does  not  reliably  produce  appropriately 
proportional  changes  in  perceived  areas.  This  problem  is  even  worse  when  adding  a  third 
dimension  to  the  graphic.  The  use  of  two  or  three  varying  dimensions  to  show  one 
dimensional  data  is  not  an  efficient  technique  and  often  leads  to  ambiguity  in  perception 
and  should  be  avoided  by  abiding  by  the  following  principle:  The  number  of 
information-carrying  (variable)  dimensions  depicted  should  not  exceed  the  number  of 
dimensions  in  the  data. 

Tufte  writes  that  to  be  truthful  and  revealing,  data  graphics  must  bear  on 
the  question  at  the  heart  of  quantitative  thinking:  “Compared  to  what?”  The  emaciated, 
data-thin  design  should  always  provoke  suspicion,  for  graphics  often  lie  by  omission, 
leaving  out  data  sufficient  for  comparisons.  The  principle:  Graphics  must  not  quote  data 
out  of  context. 

On  the  issue  of  Graphical  Integrity,  Tufte  points  out  that  graphical 
proficiency  requires  three  entirely  different  skills:  the  substantive,  statistical,  and  artistic. 
If  statistical  integrity  and  graphical  sophistication  are  to  be  achieved,  then  the  team  of 
designers  should  include  experts  in  the  area  of  substantive  and  quantitative  information. 
Don’t  leave  the  design  work  solely  up  to  the  graphic  artists  or  you  may  end  up  with 
graphics  that  (1)  lie;  (2)  employ  only  simple  designs;  (3)  and  do  not  capture  the  real  news 
of  the  data. 
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To  summarize,  in  order  to  maintain  graphical  integrity  it  is  important  for 
the  designer  to  ask  the  right  questions: 

•  Does  the  display  tell  the  truth? 

•  Is  the  representation  accurate? 

•  Are  the  data  documented? 

•  Do  the  display  methods  tell  the  truth? 

•  Are  the  appropriate  comparisons,  contrasts,  and  contexts  shown? 

2.  Theory  of  Data  Graphics 

a.  Data-Ink  and  Graphical  Redesign 

Tufte’s  fundamental  principle  of  good  statistical  graphics  is:  “Above  all 
else  show  the  data.”  This  principle  is  the  basis  for  a  theory  of  data  graphics.  The  majority 
of  ink  on  a  graphic  should  represent  data-infonnation,  with  the  ink  changing  as  the  data 
changes.  Tufte  explains  that  data-ink  is  the  non-erasable  core  of  a  graphic,  the  non- 
redundant  ink  arranged  in  response  to  variation  in  the  numbers  represented  and  should  be 
maximized  at  all  times.  The  more  of  the  graphic  that  is  devoted  to  data,  the  better  off  it 
will  be  in  displaying  that  data. 

In  conjunction  with  maximizing  the  data  to  ink  ratio,  he  also  writes  that 
one  should  erase  the  data  ink  that  is  not  representing  data.  Ink  that  doesn’t  depict 
statistical  information  doesn’t  have  much  use  to  the  viewer  of  the  graphic  and  could 
possibly  cover  up  or  clutter  up  useful  data.  One  way  to  reduce  unnecessary  data  ink  is  to 
look  at  the  bilateral  symmetry.  Is  there  a  chance  that  the  space  consumed  in  a  graphic  can 
be  halved  without  losing  any  important  infonnation?  Unless  it  serves  a  distinct  purpose, 
redundancy  of  the  same  data  should  be  avoided.  It  is  important  to  be  able  to  edit  the  data 
after  it  has  been  displayed  in  order  to  weed  out  the  unnecessary  or  redundant  data-ink  so 
that  the  critical  data  is  obviously  clear. 

These  five  principles  in  the  theory  of  data  graphics  produce  substantial 
changes  in  graphical  design.  These  principles  apply  to  many  graphics  and  yield  a  series 
of  design  options  through  cycles  of  graphical  revision  and  editing.  The  five  principles 
are: 
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•  Above  all  else  show  the  data 

•  Maximize  the  data-ink  ratio 

•  Erase  non-data  ink 

•  Erase  redundant  data-ink 

•  Revise  and  edit. 

b.  Chart  Junk:  Vibrations,  Grids,  and  Ducks 

Throughout  his  books,  Tufte  mentions  how  to  achieve  graphical 
excellence  by  avoiding  many  graphical  mistakes  or  methods  . 

Sometimes  a  designer  of  a  graphic  unintentionally  engrosses  the  body  of 
the  graphic  with  a  lot  of  ink  that  serves  no  useful  purpose  and  doesn’t  tell  the  viewer 
anything  new.  The  purpose  of  superfluous  decoration  may  have  been  to  make  the  graphic 
appear  more  scientific,  to  enliven  the  display,  or  to  give  the  designer  an  opportunity  to 
exercise  artistic  skills.  Whatever  the  reason,  the  excess  is  all  non-data-ink  or  redundant 
data-ink,  and  is  what  Tufte  refers  to  as  chart  junk.  Chart  junk  consists  of  decorative 
elements  that  provide  no  data  and  cause  confusion.  This  graphical  decoration  is  prevalent 
in  technical  publications  as  well  as  in  commercial  and  media  graphics  and  is  easier  to 
produce  than  the  hard  work  required  to  produce  intriguing  numbers  and  securing 
evidence.  Most  chart  junk  doesn’t  involve  artistic  considerations  or  motivations,  rather  it 
just  naturally  evolves  and  pervades  by  use  of  conventional  graphical  bits  and  pieces  that 
are  routinely  used  in  displays.  These  bits  and  pieces  consist  of  over-busy  grid  lines  and 
excess  ticks,  redundant  representations  of  the  simplest  data,  the  debris  of  computer 
plotting,  and  many  of  the  devices  generating  design  variation. 

Tufte  points  out  three  widespread  types  of  chart  junk  that  are  typically 
abundant  in  scientific  and  technical  research  these  days.  They  are  the  unintentional 
optical  art,  the  dreaded  grid,  and  the  graphical  duck. 

Unintentional  optical  art  is  dependent  upon  moire  effects,  in  which  the 
design  interacts  with  the  physiological  quiver  of  the  eye  to  produce  the  disturbing 
manifestation  of  vibration  and  movement.  This  disturbing  effect  extends  beyond  the  ink 
of  the  design  to  the  whole  page.  It  is  not  a  good  idea  to  draw  up  statistical  graphics  that 
appear  to  shimmer.  As  bad  as  this  moire  vibration  is  in  clouding  the  flow  of  information, 
it  is  even  more  abundant  today  due  to  the  ability  to  easily  and  quickly  create  the  ill  effects 
with  computer  programs  such  as  Excel  and  Power  Point.  “The  handbooks  and  textbooks 
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of  statistical  graphics,  along  with  the  user’s  manuals  for  computer  graphics  programs,  are 
filled  up  with  vibrating  graphics,  presented  as  exemplars  of  design”  (Tufte,  2001). 


Another  form  of  chart  junk  is  that  of  the  grid.  If  the  designer  should 
attempt  to  use  any  form  of  grid  it  should  be  very  light  or  completely  suppressed  showing 
only  an  implicit  presence  so  as  not  to  interfere  with  the  target  data.  Grids  are  used  for  the 
initial  plotting  and  should  not  show  up  on  the  final  graphic.  Grids  contain  no  information, 
tend  to  clutter  up  the  graphic,  and  generate  graphic  activity  unrelated  to  data  information. 

When  a  graphic  is  taken  over  by  decorative  forms  or  computer  debris,  the 
data  measures  and  structures  become  design  elements,  or  the  overall  design  purveys 
graphical  style  rather  than  quantitative  information,  then  that  graphic  may  be  called  a 
duck  in  honor  of  the  duck-form  store,  “Big  Duck.”  For  this  building  the  whole  structure 
is  itself  decoration,  just  as  in  the  duck  data  graphic.  (Tufte,  2001) 

Designers  of  data  graphics  should  seek  to  use  new  technology  in  a  way 
that  present  data  in  a  meaningful  way  rather  than  quickly  and  massively  spitting  out 
graphics  that  just  look  cool.  Chart  junk  can  turn  a  mediocre  graphic  or  chart  into  a  total 
disaster,  but  it  can  never  replace  or  effectively  represent  a  thin  data  set.  A  designer  should 
never  use  chart  junk,  including  moire  vibration,  the  grid,  or  the  duck. 

c.  Data-Ink  Maximization  and  Graphical  Design 
By  using  the  previous  principles,  it  is  possible  to  not  only  edit  old  designs 
but  derive  new  graphical  forms  as  well.  Tufte  uses  several  examples  where  he  shows 
before  and  after  effects  of  simple  graphs  to  show  that  his  theory  of  data  graphics  is 
correct.  (See  Figure  5).  In  some  of  his  examples,  the  observed  increases  in  efficiency,  or 
how  much  of  the  graphic’s  ink  carries  information,  were  sometimes  quite  large.  In 
several  cases,  the  data-ink  ratio  increased  from  .1  or  .2  to  nearly  1.0.  These  transformed 
designs  were  less  cluttered  and  could  be  shrunk  down  more  easily  than  the  originals.  But 
are  these  transformed  ideas  better?  Tufte  answers  this  question  by  saying: 
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•  They  are  better  within  the  principles  of  the  theory,  for  more  information  per 
unit  of  space  and  per  unit  of  ink  are  displayed.  It’s  a  significant  gain  when  you 
consider  that  the  history  of  devices  for  communicating  information  is  written 
in  terms  of  increases  in  efficiency  of  communication  and  production. 

•  Graphics  are  almost  always  going  to  improve  as  they  go  through  editing, 
revision,  and  testing  against  different  design  options.  The  principles  of 
maximizing  data-ink  and  erasing  generate  graphical  alternatives  and  also 
suggest  a  direction  in  which  revisions  should  move. 

•  Then  there  is  the  audience:  will  those  looking  at  the  new  designs  be  confused? 
Some  of  the  designs  are  self-explanatory.  The  chance  of  a  member  of  the 
audience  being  confused  is  remote.  Nothing  is  lost  to  those  who  are  puzzled 
by  the  changes  and  something  is  gained  by  those  who  do  understand  it. 
Moreover,  it  is  a  frequent  mistake  in  thinking  about  statistical  graphics  to 
underestimate  the  audience. 

•  Some  of  the  new  designs  may  appear  odd,  but  this  is  probably  because  we  have 
not  seen  them  before.  The  conventional  designs  for  statistical  graphics  that 
have  been  viewed  thousands  of  times  where  as  the  new  designs  have  not.  In 
time  and  with  use  the  new  designs  would  become  just  as  reasonable  as  the  old. 
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Figure  5.  Before  and  after  versions  of  numerical  charts.  (From  Salon) 
d.  Multi-functioning  Graphical  Elements 

The  same  ink  should  often  serve  more  than  one  graphical  purpose  at  a 
time.  A  graphical  element  may  carry  data  information  and  also  perform  a  design  function 
usually  left  to  non-data-ink.  Or  it  might  show  several  different  pieces  of  data.  Such  multi¬ 
functioning  graphical  elements,  if  designed  with  care  and  subtlety,  can  effectively  display 
complex,  multivariate  data.  (Tufte.  2001)  The  principle  is:  Mobilize  every  graphical 
element,  perhaps  several  times  over,  to  show  the  data. 

In  other  words,  use  the  graphic  segment  to  show  or  portray  more  than  one 
item  of  data.  Graphical  structures  can  often  serve  several  purposes  at  once.  For  example, 
the  stem  and  leaf  plots  display  sequences  of  numbers  which  directly  portray  structure  by 
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the  physical  length  of  each  sequence.  Tufte  contends  that  the  designers  of  such  graphical 
elements  must  be  careful  because  multi-functioning  elements  tend  to  generate  graphical 
puzzles,  with  encoding  that  only  the  designer  can  figure  out.  The  designers  must  strive 
for  enhancing  graphical  clarity  in  the  face  of  complexity  when  developing  multi¬ 
functioning  elements. 

Those  graphical  elements  that  actually  locate  or  plot  data  in  a  graph  or 
chart  are  called  the  data  measure.  For  example,  the  bars  of  a  bar-chart,  the  dots  of  a 
scatter-plot,  the  dots  and  dashes  of  a  dot-dash-plot,  or  the  blots  of  a  blot  map  are  data 
measures.  The  data  grid  itself  may  be  the  data,  revealing  both  the  values  and  the 
coordinate  system  at  the  same  time.  The  ink  that  makes  up  the  data  measure  can  itself 
carry  data,  for  example,  the  dots  of  the  scatter-plot  can  take  on  different  shading  in 
response  to  a  third  variable.  This  rule  can  also  be  used  with  data-based  coordinate  lines  as 
well.  Why  not  use  that  ink  to  show  data  in  the  form  of  data-based  labels? 

In  many  graphics  the  coordinate  labels  are  separated  from  the  data 
measures.  Because  of  that  the  eye  of  the  viewer  must  move  back  and  forth  between  the 
path  formed  by  the  data  and  the  coordinate  positions  arrayed  along  the  margins  of  the 
graphic.  It  is  possible  that  this  eye  work  could  be  eliminated  entirely  by  turning  the 
coordinate  labels  into  data  measures,  another  double  functioning  maneuver. 

e.  Colors 

Much  of  Tufte’s  books  talk  about  the  effect  that  colors  have  on  graphics. 
However,  much  of  what  he  writes  about  does  not  apply  directly  towards  the  graphics  that 
we  are  concerned  with  in  this  study  so  the  we  have  tried  to  compile  only  the  relevant 
applications  of  using  color  as  they  may  apply  in  the  redesign  of  the  promotion  displays. 

Of  course  color  brings  to  information  more  than  just  codes  naming  visual 
nouns  -  color  is  a  natural  quantifier,  with  a  perceptually  continuous  (in 
value  and  saturation)  span  of  incredible  fineness  of  distinction,  at  a 
precision  comparable  to  most  measurement  (Tufte,  1990). 

Color  use  can  greatly  enhance  data  comprehension  and  can  convey  multi¬ 
dimensional  values  effectively.  Color  grids  are  a  form  of  layering  which  provides  context 
but  which  should  be  unobtrusive  and  muted.  Pure  bright  colors  should  be  reserved  for 
small  highlight  areas  and  almost  never  used  as  backgrounds.  Colors  can  be  use  as  labels, 
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as  measures,  and  to  imitate  reality  as  in  the  blue  lakes  in  maps.  Color  spots  against  a  light 
gray  are  also  effective. 

Despite  these  benefits  of  using  colors  in  displaying  information,  the 
designer  must  avoid  using  colors  in  a  way  that  generates  graphical  puzzles.  Despite  our 
experiences  with  the  spectrum  in  science  textbooks  and  rainbows,  the  mind’s  eye  does 
not  readily  give  a  visual  ordering  to  colors,  except  possibly  for  red  to  reflect  higher  levels 
than  other  colors  (Tufte  2001).  Attempts  to  give  colors  an  order  often  requires  the  viewer 
to  mentally  decode  the  color  scheme,  trying  to  recall  what  the  different  colors 
represented,  and  to  look  back  at  the  legend  to  clarify  it. 

The  success  of  gray  compared  to  the  visually  more  spectacular  color  gives 
us  a  lead  on  how  multi-functioning  graphical  elements  can  communicate  complex 
information  without  turning  into  puzzles.  The  shades  of  gray  provide  an  easily 
comprehended  order  to  the  data  measures.  This  is  the  key.  Central  to  maintaining  clarity 
in  the  face  of  the  complex  are  graphical  methods  that  organize  and  order  the  flow  of 
graphical  infonnation  presented  to  the  eye. 

So  what  colors  should  we  use  in  order  to  represent  and  illuminate 
information?  Tufte  suggests  the  use  of  colors  found  in  nature,  especially  those  on  the 
lighter  side,  such  as  blues,  yellows,  and  grays  of  sky  and  shadow. 

Nature’s  colors  are  familiar  and  coherent,  possessing  a  widely  accepted 
hannony  to  the  human  eye  -  and  their  source  has  a  certain  definitive 
authority.  A  palette  of  nature’s  colors  helps  suppress  production  of  garish 
and  content-empty  color  junk.  Local  emphasis  for  data  is  then  given  by 
means  of  spot  highlights  of  strong  color  woven  through  the  serene 
background  (Tufte,  1990). 

In  their  book  Visualization:  Using  Computer  Graphics  to  Explore  Data 
and  Present  Information,  the  authors  explain  that  in  displays  of  graphs  and  charts,  there 
are  overlapping  lines,  grids,  background,  and  text.  Thus  a  designer  needs  to  select  colors 
so  that  the  various  pieces  of  information  are  distinguishable  even  when  they  are 
overlapping.  “The  connotation  of  color  is  critical  in  monitoring  applications  because  the 
viewer  must  react  quickly  to  the  appearance  of  a  particular  color  in  a  given  location.” 
(Brown,  et  al.  1995)  Table  3  and  Table  4  reproduced  from  their  book  show  the  best  and 
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worst  color  combinations,  as  determined  from  a  study  where  at  least  four  of  sixteen 
participants  accepted  or  rejected  each  color  combination. 


Best  Color  Combinations 
(N  =  16) 


Background 

Thin  Lines  and  Text 

Thick  Lines  and  Panels 

White 

Blue  (94%),  Black  (63%),  Red  (25%) 

Black  (69%),  Blue  (63%),  Red  (31%) 

Black 

White  (75%),  Yellow  (63%) 

Yellow  (69%),  White  (50%), 

Green  (25%) 

Red 

Yellow  (75%),  White  (56%),  Black  (44%) 

Black  (50%),  Yellow  (44%), 

White  (44%),  Cyan  (31%) 

Green 

Black  (100%),  Blue  (56%),  Red  (25%) 

Black  (69%),  Red  (63%),  Blue  (31%) 

Blue 

White  (81%),  Yellow  (50%),  Cyan  (25%) 

Yellow  (38%),  Magenta  (38%), 

Black  (31%),  Cyan  (31%),  White  (25%) 

Cyan 

Blue  (69%),  Black  (56%),  Red  (37%) 

Red  (56%),  Blue  (50%),  Black  (44%), 
Magenta  (25%) 

Magenta 

Black  (63%),  White  (56%),  Blue  (44%) 

Blue  (50%),  Black  (44%),  Yellow  (25%) 

Yellow 

Red  (63%),  Blue  (63%),  Black  (56%) 

Red  (75%),  Blue  (63%),  Black  (50%) 

Table  3 

Best  Color  Combination.  (From  Brown,  et  al.) 

Worst  Color  Combinations 
(N=  16) 

Background 

Thin  Lines  and  Text 

Thick  Lines  and  Panels 

White 

Yellow  (100%),  Cyan  (94%) 

Yellow  (94%),  Cyan  (75%) 

Black 

Blue  (87%),  Red  (37%),  Magenta  (25%) 

Blue  (81%),  Magenta  (31%) 

Red 

Magenta  (81%),  Blue  (44%), 

Green  (25%),  Cyan  (25%) 

Magenta  (69%),  Blue  (50%), 

Green  (37%),  Cyan  (25%) 

Green 

Cyan  (81%),  Magenta  (50%), 

Yellow  (37%) 

Cyan  (81%),  Magenta  (44%), 

Yellow  (44%) 

Blue 

Green  (62%),  Red  (37%),  Black  (37%) 

Green  (44%),  Red  (31%),  Black  (31%) 

Cyan 

Green  (81%),  Yellow  (75%),  White  (31%) 

Yellow  (69%),  Green  (62%), 

White  (56%) 

Magenta 

Green  (75%),  Red  (56%),  Cyan  (44%) 

Cyan  (81%),  Green  (69%),  Red  (44%) 

Yellow 

White  (81%),  Cyan  (81%) 

White  (81%),  Cyan  (56%),  Green  (25%) 

Table  4 

Worst  Color  Combinations.  (From  Brown,  et  al.) 

f.  Data  Density  and  Small  Multiples 

Graphics  are  best  when  they  represent  very  dense  and  rich  data  sets. 
Within  a  small  area,  our  eyes  can  make  an  extraordinary  number  of  distinctions.  For 
example,  by  using  very  light  grid  lines  it  is  easy  to  locate  625  points  in  one  square  inch, 


35 


or  equivalently,  100  points  in  one  square  centimeter.  It  would  be  wise  then  in  the  design 
of  statistical  graphics  to  take  advantage  of  the  eye’s  ability  to  detect  large  amounts  of 
information  in  small  spaces.  But  how  does  one  gauge  how  much  information  the  graphics 
should  show?  Tufte  calls  this  the  measure  of  data  density.  Taking  into  account  the  size  of 
the  graphic  in  relation  to  the  amount  of  data  it  displays  yields  the  data  density: 

Data  density  =  (no.  of  entries  in  data  matrix)/(area  of  graphic) 

Figure  6  of  the  New  York  City  weather  history  is  a  good  example  of  a 
graphic  with  a  high  data  density  of  181  numbers  per  square  inch  (28  per  square 
centimeter). 


Figure  6.  New  York  Weather  History  (From  Tufte) 


Data  graphics  should  be  based  on  large  rather  than  small  data  matrices  and 
have  a  high  rather  than  low  data  density.  More  infonnation  is  better  than  less  information, 
especially  when  the  marginal  costs  of  using  and  interpreting  additional  information  are 
small,  as  they  are  for  most  graphics.  Tufte  also  recommends  that  the  simple  items  should 
left  out  in  tables  or  in  the  text.  Graphics  can  give  a  sense  of  large  and  complex  data  sets 
that  cannot  be  managed  in  any  other  way.  If  it  becomes  overcrowded  (although  several 
thousand  numbers  represented  isn’t  usually  a  problem)  a  variety  of  data  reduction 
techniques  -  averaging,  clustering,  smoothing  -  can  thin  the  numbers  out  before  plotting. 
It  is  important  to  realize  that  summary  graphics  can  emerge  from  high-infonnation 
displays,  but  there  is  nowhere  to  go  if  we  begin  with  a  low-information  design. 

Graphical  designs  that  are  data-  rich  give  context  and  credibility  to 
statistical  evidence.  Viewers  may  question  low-information  designs  by  wondering  what  is 
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left  out,  what  is  hidden,  and  why  are  we  shown  so  little.  Graphics  that  are  high  density 
will  help  the  viewer  to  compare  parts  of  the  data  by  displaying  much  information  within 
the  view  of  the  eye.  The  principle  to  remember  here  is:  Maximize  data  density  and  the 
size  of  the  data  matrix,  within  reason. 

One  must  be  careful  when  designing  high-  infonnation  graphics  because 
as  the  volume  of  data  increases,  data  measures  must  shrink  (smaller  dots  for  scatter  plots 
or  thinner  lines  for  time  series,  etc.)  The  appearance  and  disorder  of  chart  junk,  non-data- 
ink,  and  redundant  data-ink  can  result  in  even  greater  disaster  in  data-rich  designs.  The 
way  to  increase  data  density  without  having  to  enlarge  the  data  matrix  is  to  reduce  the 
area  of  the  graphic.  Tufte  points  out  that  many  data  graphics  can  be  reduced  in  area  to 
half  their  currently  published  size  with  virtually  no  loss  in  legibility  and  information. 

In  review,  good  graphics  are: 

•  Comparative 

•  Multivariate 

•  High  density 

•  Able  to  reveal  interactions,  comparisons,  etc. 

•  Drawn  almost  entirely  with  data-ink 

g.  Aesthetics 

Tufte  writes  that  graphical  elegance  consists  of  design  simplicity  , 
complexity,  and  truth  of  data.  Graphics  that  are  visually  striking  generate  power  from 
content  and  interpretations  beyond  the  immediate  display  of  some  numbers.  To  attain 
attractive  displays  of  statistical  infonnation  a  graphic  should: 

•  Have  a  properly  chosen  format  and  design  . 

•  Use  words,  numbers,  and  drawing  together. 

•  Reflect  a  balance,  a  proportion,  and  a  sense  of  relevant  scale. 

•  Display  an  accessible  complexity  of  detail. 

•  Often  have  a  narrative  quality;  a  story  to  tell  about  the  data. 

•  Be  drawn  in  a  professional  manner,  with  the  technical  details  if  production 
done  with  care. 

•  Avoid  content-free  decoration,  including  chart  junk. 

Tufte  explains  that  words  and  pictures  belong  together  and  sometimes 
viewers  need  help  that  words  can  provide.  It  is  almost  always  helpful  to  write  little 
messages  on  the  plotting  field  to  explain  the  data,  to  label  the  outliers  and  interesting  data 
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points,  to  write  equations  and  sometimes  tables  on  the  graphic  itself,  and  to  integrate  the 
captions  and  legend  into  the  design  so  that  the  eye  is  not  required  to  dart  back  and  forth 
between  textual  material  and  the  graphic.  The  size  of  type  on  and  around  graphics  can  be 
quite  small,  since  the  phrases  and  sentences  are  usually  not  too  long  -  and  therefore  the 
small  type  will  not  fatigue  viewers  the  way  it  does  in  lengthy  texts.  The  principle  of 
data/text  integration  is:  Data  graphics  are  paragraphs  about  data  and  should  be  treated  as 
such. 

When  graphical  elements  are  together  and  their  proportions  are  in  balance, 
it  looks  better  and  ensures  that  an  integrated  quality  and  appropriate  visual  linkage  exists 
between  them.  Lines  in  data  graphics  should  be  thin.  The  orthogonal  intersection  of  lines 
of  different  weights  is  an  effective  aesthetic  device  as  well. 

The  design  of  data  graphics  should  tend  toward  the  horizontal  rather  than 
the  vertical.  Tufte  believes  that  a  display  greater  in  length  than  height  has  advantages  due 
to  the  fact  that  our  eye  is  naturally  practiced  in  detecting  deviations  from  the  horizon,  and 
graphic  design  should  take  advantage  of  this  fact.  Horizontally  stretched  time  series  are 
more  accessible  to  the  eye.  The  analogy  to  the  horizon  also  suggests  that  a  shaded,  high 
contrast  display  might  occasionally  be  better  that  the  non-shaded  lines  that  resemble  a 
floating  snake.  It  is  also  easier  to  read  words  that  read  from  left  to  right  on  a  horizontally 
stretched  plotting  field.  A  longer  horizontal  axis  helps  to  elaborate  the  workings  of  the 
causal  variable  in  more  detail.  How  much  wider  should  it  be?  Tufte  writes  that  a  good 
design  would  move  toward  the  horizontal  about  50  percent  wider  than  tall. 

3.  Conclusion 

The  importance  of  a  well  thought-out,  and  coherent  design  of  the  graphics  for  the 
promotion  visual  displays  cannot  be  overstated.  The  rapid  and  accurate  transfer  of 
information  by  use  of  sound  graphical  principles  can  have  a  profound  effect  on  future  of 
the  Navy’s  promotion  system.  In  determining  the  design  of  choice,  the  principles  set  forth 
above  should  help  generate  design  options  and  guide  choices  among  options.  It  is  Tufte ’s 
belief  that  these  principles  should  be  applied  flexibly  because  they  are  not  logically  or 
mathematically  certain  and  it  would  be  better  to  violate  any  principle  than  to  place 
graceless  or  inelegant  marks  on  paper. 
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What  is  to  be  sought  in  design  for  the  display  of  information  is  the  clear 
portrayal  of  complexity.  Not  the  complication  of  the  simple;  rather  the 
task  of  the  designer  is  to  give  visual  access  to  the  subtle  and  the  difficult  - 
that  is,  the  revelation  of  the  complex  (Tufte,  2001). 
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IV.  DATA  COLLECTION  AND  ANALYSIS 


A.  INTRODUCTION 

The  proof-of-concept  (POC)  framework  for  this  study  is  the  Navy’s  current 
officer  promotion/selection  process.  In  order  to  answer  the  principle  research  question  we 
have  developed  a  POC  prototype  visualization  solution  through  the  use  of  COTS 
software  and  technology  in  order  to  display,  at  a  minimum,  visualization  concepts  and 
capabilities  currently  available.  This  prototype  uses  a  simple  notional  relational  database 
(Microsoft  Access)  and  simulates  the  current  system’s  input  and  outputs.  Using  three-tier 
architecture,  the  prototype  will  extract  the  data  from  the  database  and  display  an 
improved  visual  display  for  the  board  to  interpret  and  compare.  Additionally,  because  the 
system  is  web-  based,  this  data  would  also  be  available  to  the  individual  briefer  in  order 
to  prepare  his  or  her  records  for  the  tank  briefings.  We  propose  that  by  utilizing  a  web- 
based  system  such  as  our  POC  and  allowing  electronic  documents  to  be  produced  and 
shared  via  the  network  the  new  system  will  substantially  reduce  the  time  and  increase  the 
effectiveness  of  the  promotion  process.  In  order  to  provide  evidence  of  this,  we  have 
created  the  following  sections: 

•  An  “as-is”  analysis  with  a  Return  on  Knowledge  (ROK)  spreadsheet 
and  graphical  representation. 

•  A  “to-be”  incremental  model  spreadsheet. 

•  Comparison  of  the  “as  is”  and  “to  be”  models  with  KVA  analysis. 

•  A  fully  functional  web  site  that  utilizes  a  notional  relational  database. 

•  Prototype  web-based  charts  and  graphs  using  different  colors,  along 
with  layering  and  separation  multidimensional  schemes  that  would 
facilitate  infonnation  flow. 

The  focus  of  the  chapter  will  be  on  the  Knowledge  Value  Added  methodology 
which  will  provide  us  with  a  perfonnance  ratio  for  all  the  core  processes  involved  in 
order  to  determine  if  there  is  any  increase  in  effectiveness  between  the  “as-is”  and  “to- 
be”  models. 

B.  DATA  COLLECTION 

Our  data  collection  process  focused  on  obtaining  the  most  suitable  information 
that  would  provide  us  with  the  necessary  background  to  answer  the  research  questions. 
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Here  we  will  explain  how  we  collected  the  data.  The  knowledge  assessment  -  associated 
with  the  processes  identified  during  our  data  collection  will  identify  opportunities  for 
increased  returns.  The  scope  of  our  data  collection  was  limited  to  the  officer  promotion 
process  starting  from  the  pre-board  stage,  beginning  with  uploading  the  list  of  eligible 
officers,  up  until  the  final  Tentatively  Selected  (TS)  list  is  produced.  By  following  the 
seven  steps  of  the  KVA  process  as  listed  in  Table  1  of  chapter  3,  we  were  able  to 
calculate  the  ROK  for  the  “as  is”  promotion  process. 

1.  Step  One:  Identify  Core  Process  and  Sub-processes 

Our  first  step  was  to  identify  the  core  process  and  its  sub-processes.  In  order  to 
establish  the  boundaries  of  the  core  process  we  needed  to  describe  in  detail  the  “as  is” 
process: 

•  Identify  the  end  output  of  the  core  process. 

•  Identify  the  sub-process  outputs  that  eventually  create  the  end  product. 

•  Create  and  aggregate-level  process  diagram  that  depicts  the  inputs  and  outputs 
of  the  core  and  each  of  its  sub-processes.  Figure  7  is  a  depiction  of  our 
workflow  model. 

The  information  that  was  collected  on  the  “as  is”  process  was  gathered  through 
interviews  with  the  subject  matter  experts  (SMEs)  involved  in  the  overall  process.  The 
SMEs  were  asked  to  break  the  overall  process  down  into  smaller,  more  specific  sub¬ 
processes  through  which  we  could  obtain  average  learning-time  estimates  and  the  number 
of  roughly  equivalent  process  instructions  required  to  complete  each  sub-process.  From 
this  data  we  were  able  to  break  up  the  core  promotion  process  into  the  following  five 
separate  and  distinct  processes  in  which  to  base  our  “as  is”  base  line  model: 

•  IT  Review 

•  Assistant  Recorder  Review 

•  Briefer  Review/ Annotation 

•  Data  Downloads 

•  Brief/Presentation/Vote 
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Data 


Figure  7.  “As  Is”  Process  Model 

a.  IT  Review 

The  first  step  in  the  typical  promotion  board  process  starts  about  four 
months  before  the  convening  date.  This  is  when  the  initial  list  of  eligible  officers  is 
compiled  and  modified  as  required.  The  eligible  list  is  continually  synchronized  with  an 
official  automated  database  to  ensure  consideration  of  all  candidates.  The  master  file  is 
queried  six  weeks  prior  to  the  board  convening  for  FITREP  continuity  and  messages  are 
sent  for  those  that  are  missing.  The  specific  process  instructions  were  identified  as: 
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•  Upload  list  of  eligible. 

•  Perform  FITREP  continuity  check. 

•  Create  a  discrepancy  database  of  record  with  gaps  greater  than  90  days. 

•  Post  list  of  web  for  Fleet  access. 

•  Obtain  UICs  for  command’s  members  missing  FITREPS. 

•  Send  messages  to  Fleet  notifying  members  of  gaps  in  records. 

•  Log  all  faxed  FITREPS  received  and  check  for  validity. 

•  Update  website  list  of  missing  FITREPS. 

•  If  it’s  a  valid  FITREP,  then  send  to  the  system  scanner;  if  not  a  valid  FITREP, 
then  notify  command. 

•  Prepare  board  room  and  tank  for  use. 

•  Provide  board  members  and  recorders  with  welcome  aboard  packages. 

b.  Assistant  Recorder  Review 

For  most  boards  the  recorders  arrive  one  week  prior  to  the  board  actually 
being  convened.  During  this  week,  the  assistant  recorders  are  trained  in  the  board's 
administrative  procedures  and  assigned  approximately  80  service  records  to  review.  Once 
proficient,  a  service  record  review  takes  15  to  20  minutes. 

This  is  the  first  time  that  someone  other  than  the  eligible  will  review  the 
record  as  it  will  appear  before  the  selection  board.  The  record  is  reviewed  for  several 
items  including  regular  report  continuity  for  the  last  five  years.  The  OSR/PSR  is  checked 
to  see  that  FITREP  grades  are  accurate  and  the  record  is  reviewed  to  ensure  that  award 
citations  are  present  for  the  number  and  type  of  awards  that  are  listed  on  the  OSR/PSR. 
The  assistant  recorder  also  verifies  that  an  eligible  has  a  photo  in  current  grade. 

Any  documentation  the  board  receives,  either  directly  from  the  service 
member  or  through  the  detailer,  is  logged  into  the  system  and  scanned  into  the  selection 
board  copy  of  the  service  record.  Only  then  can  the  assistant  recorder  make  the  necessary 
changes  on  the  appropriate  Summary  Cover  Sheet  and  the  PSR.  The  process  instructions 
involved  include: 
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•  Electronically  select  and  open  record  for  review. 

•  Check  that  the  number  of  awards  in  written  comments  is  the  same  in  OSR/PSR 
page. 

•  Check  that  the  education  in  written  comments  is  the  same  as  on  the  OSR/PSR. 

•  Check  that  the  grades  on  FITREPS  are  the  same  as  on  the  OSR/PSR. 

•  Check  that  photo  is  current. 

•  Prepare  a  list  of  discrepancies. 

•  Send  messages  to  member’s  commands  with  missing  data. 

•  Accept  and  enter  late  data. 

•  Send  to  the  board  members  administrative  requirements. 

c.  Briefer  Review/Annotation 

This  sub-process  begins  once  the  actual  board  convenes.  Basically,  once 
the  board  members  are  checked  in,  they  will  receive  the  administrative  and  welcome 
aboard  briefs.  They  will  then  be  briefed  on  the  board  precept;  a  document  that  is  signed 
by  the  Secretary  of  the  Navy  and  is  delivered  to  the  board  president.  It  gives  guidance  on 
how  to  conduct  the  board  and  is  written  each  time  there  is  a  promotion  board.  The 
members  will  then  be  given  a  number  of  records  to  review,  annotate,  and  grade  in 
preparation  for  the  presentation  to  the  rest  of  the  board.  The  process  instructions  include: 

•  Review  Precept  letter: 

-  selection  standard 

-  authorized  percentage  to  select 

-  cause  detennination 

-  general  procedures  and  schedule 

-  equal  opportunity  guidelines 

-  results  management  guidelines 

•  Select  records  for  review 

•  Review  and  annotate  exceptional  performance  in  operational  environment, 
command  leadership,  integrity,  resourcefulness,  and  innovation. 

•  Review  and  annotate  graduate  education/specialty  training. 

•  Review  and  annotate  clinical  proficiency/skill,  management  skills. 

•  Review  and  annotate  innovation  and  retention  effectiveness. 

•  Review  and  annotate  antiterrorism/force  protection  qualifications. 

d.  Data  Download 

After  the  individual  records  have  been  reviewed,  annotated,  and  graded, 
the  reports  are  then  saved  to  a  disk  and  then  separately  downloaded  into  the  selection 
board  voting  tank  module.  This  procedure  often  results  in  a  substantial  slow  down  in  the 
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overall  process  because  the  information  is  not  immediately  available  to  the  voting 
members  in  the  tank.  For  example,  once  enough  records  have  been  reviewed  and  a  tank 
is  available  (often  there  is  more  than  one  board  in  session  so  the  tanks  may  be 
unavailable)  the  senior  member  may  decide  to  go  to  the  tank  for  the  vote.  This  may  be 
delayed  because  the  reviewed  records  now  have  to  be  separately  downloaded  into  the 
voting  tank  module.  The  sub-process  includes: 

•  Save  annotated  data  file  to  disk. 

•  Forward  data  file  for  manual  download  to  the  board  voting  tank. 

e.  Brief/Presentation/Vote 

After  the  records  have  been  downloaded  into  the  tank,  all  of  the  voting 
members  gather  in  the  tank  where  the  member  who  reviewed  the  record  briefs  it  while 
the  eligible  individual’s  OSR/PSR  and  cover  page  are  displayed  and  all  board  members 
vote.  Sub-process: 

•  Briefer  displays  and  orally  reviews  annotated  data. 

•  All  members  compare  with  experience  baseline  for: 

-  Exceptional  perfonnance  in  operational  environment,  command 
leadership,  integrity,  resourcefulness,  and  innovation 

-  Graduate  education/specialty  training 

-  Clinical  proficiency/skill,  management  skills 

-  Innovation  and  retention  effectiveness 

-  Anti-terrorism/force  protection  qualifications 

•  All  members  cognitively  “normalize”  graded  averages  and  the  R.S.’s 
cumulative  average. 

•  All  members  cast  vote  for  each  record,  cognitively  nonnalizing  all  data  with 
regard  to  all  eligible  records. 

2.  Step  two:  Establish  Common  Units  to  Measure  Learning  Time 

In  step  two,  we  were  able  to  establish,  through  our  interviews  with  the 
SMEs,  a  common  definition  of  learning  time  (LT)  for  the  five  processes  under  review. 
LT  is  the  amount  of  time  it  takes  for  an  average  person  to  learn  how  to  complete  the 
process  correctly  and  is  directly  correlated  to  the  amount  of  knowledge  embedded  in  a 
given  process. 

3.  Step  three:  Calculate  Learning  Time  to  Execute  each  Sub-process 

With  the  SMEs  input,  we  were  able  to  determine  the  total  time  it  would  take  to 


learn  how  to  execute  each  sub-process.  Given  that  there  were  an  estimated  total  of  100 
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weeks  to  learn  how  to  execute  the  five  processes;  the  distribution  of  learning  time  is 
depicted  in  Table  5  below. 


Process: 

Learning  time 
(months) 

Data  Download 

1 

IT  Review 

6 

Assistant  Recorder  Review 

8 

Briefer  Review/ Annotation 

41 

Brief/Presentation/Vote 

44 

Total 

100 

Table  5  Actual  Learning  Times 


4.  Step  four:  Designate  Sampling  Time  Period 

In  step  four,  instead  of  designating  a  sampling  time  period  in  order  to  capture  a 
representative  sample  of  the  compound  processes’  final  output,  we  used  the  number  of 
employees  within  each  core  area  to  determine  the  weighted  amount  of  knowledge 
executed  in  the  process  per  1000  records.  The  “Persons  involved”  category  reflects  the 
number  of  people  involved  in  completing  a  process.  Accounting  for  the  number  of 
employees  gives  a  general  idea  of  how  often  knowledge  (K)  is  fired  and  provides  a 


rough-cut  way  of  weighting  knowledge  in  the  processes  over  the  evaluation  period 
(Housel  and  Bell,  2001).  See  Table  6. 


Value  added 
(hours  to  train) 

Persons  involved 
(weighting  factor) 

Time 

subtotal 

Data  Download 

.2 

1 

.2 

IT  Review 

.5 

3 

1.5 

Assistant  Recorder  Review 

.8 

7 

5.6 

Briefer  Review/ Annotation 

3 

1 

3 

Brief  /presentation/Vote 

5 

11 

55 

Totals: 

9.5 

23 

65.3 

Table  6  Step  4:  Weighting  factor 


5.  Step  five:  Multiply  the  Learning  Time  for  each  Sub-Process 

In  this  case  we  took  the  time  subtotal  and  multiplied  it  by  the  number  of  times  the 

knowledge  for  that  particular  sub-process  was  fired  or  actually  used.  The  number  of 

times  fired  is  a  close  approximation  from  the  SME  in  that  field  of  how  many  times  per 

1000  records  that  the  requisite  knowledge  for  that  process  was  actually  used.  We  then 
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used  that  subtotal  and  multiplied  it  by  the  estimated  number  of  iterations  of  the  sub¬ 
process  to  be  executed  during  a  board  for  each  1000  records.  The  result  is  the  weighted 
value  which  is  ultimately  used  as  the  top  half  of  the  return  ratio.  In  other  words,  the  sum 
of  the  total  amount  of  learning  time  knowledge  for  each  process  is  used  as  the  numerator 


in  the  ROK  equation.  It  is  the  knowledge  required  to  get  the  outputs.  See  Table  7  below. 


Time 

Subtotal 

X  #  of 

times 

fired 

Fired 

subtotal 

X  #  of 
iterations 

=  Weighted 
Value  Added 

Data  Download 

.2 

1 

.2 

6 

1.2 

IT  Review 

1.5 

1 

1.5 

1 

1.5 

Assistant/Recorder 

5.6 

143 

800.8 

1 

800.8 

Brief  Review 

3 

83 

249 

2 

498 

Brief//presentation 

55 

1000 

55000 

2 

111000 

Total: 

65.3 

111301.5 

Table  7  Step  5  of  the  KVA  Calculation 


6.  Step  6:  Allocate  Revenue 

In  step  six  we  calculate  the  cost  to  execute  each  sub-process  based  solely  on  the 
cost  of  the  personnel  at  each  sub-process.  Rather  than  trying  to  figure  out  the  different 
TAD  (Temporary  Additional  Duty)  costs  for  each  rank,  which  is  too  complex,  we  chose 
to  normalize  the  cost  data  in  the  following  way.  If  the  process  used  two  E-5  personnel 
then  we  estimated  how  long  in  hours  it  took  to  complete  the  process  step  and  then 
determined  the  cost  based  on  the  average  time  of  service  of  a  typical  E-5  in  the  Navy. 
For  example,  an  E-5  over  six  years  of  service  gets  paid  X  dollars  per  month  divided  by  80 
hours  per  month.  We  then  determined  that  if  the  process  step  took  eight  man-hours  then 
eight  man-hours  times  the  hourly  rate  equals  the  total  sub-process  cost.  Here  we 
attempted  to  isolate  just  those  costs  that  pertain  to  the  creation  of  the  process  outputs. 
The  estimates  were  based  on  the  SMEs  approximations  on  how  long  it  took  individuals  to 
generate  the  outputs  necessary  to  complete  a  specific  task.  For  example,  referring  to  table 
xxx,  we  estimated  that  for  every  1000  records  reviewed,  six  separate  data  downloads 
were  required  on  average,  based  on  160  records  reviewed  before  being  sent  to  the  tank. 
We  estimated  that  each  download  would  take  about  1  hour  for  a  total  execution  time  for 
that  process  of  six  hours  per  1000  records  reviewed.  See  Table  8. 
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Execution  time  (hours) 

Process  Costs  ($) 

Data  Download 

6 

$40.70 

IT  Review 

72 

$366.30 

Assistant/Recorder 

280 

$1852.80 

Brief  Review 

360 

$27,633.59 

Brief//presentation 

120 

$29,655.56 

Total: 

$59,548.95 

Table  8  Step  6  of  the  KVA  Calculation 


7.  Step  7:  Calculate  ROK  and  Interpret  the  Results 

In  the  last  step  we  computed  the  ROK  for  each  sub-process  by  taking  the 
weighted  value  added  for  each  sub-process  and  dividing  it  by  the  process  costs.  By 
breaking  the  ROK  figures  down  to  the  individual  processes,  we  are  then  able  to  generate 
an  order  of  magnitude  estimate  of  the  return,  based  solely  on  the  ratio  of  cost  to  amount 
of  knowledge  and  process  instructions  executed  during  the  sample  period.  The  overall 
ROK  for  all  of  the  processes  in  the  “as  is”  model  is  186.91%.  See  Table  9. 


=  Weighted 

Piocess 

Execution  time 

Piocess  Costs 

ROK*'* 

i/.i  hie  Added  1  inductions  uivsti 

(fours) 

C5) 

(%) 

1.2 

2 

6 

$40.70 

2.95% 

1.5 

11 

72 

$366.30 

0.41% 

800.8 

9 

280 

$1 ,852.80 

43.22% 

498 

13 

360 

$27,633.59 

1.80% 

110000 

48 

120 

$29,655.56 

370.93% 

111301 .5 

83 

$59,548.95 

186.91% 

Table  9  ROK  Calculations 
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VISUALIZATION 

SELECTION  BOARD  PROCESS 

i  PER  1000  RECORDS) 

Sub  P ioc  ess 

Learning  time 

Value  added 

Persons 

Time 

X  not 

Filed 

X  =  of 

=  Weitjrted 

Piocess 

Execution  time 

Piocess  Costs 

ROK'" 

(m  orths) 

(hours  to  train*) 

Involved 

Subtotal 

Times  Filed' 

Subtotal 

s 

V 

I 

Value  Added 

Insd  notion  sruwtr 

(hours) 

CD 

<%) 

Data  Download 

1 

0.2 

1 

0.2 

1 

0.2 

6 

1.2 

2 

6 

140.70 

2.95% 

IT  Review 

6 

0.5 

3 

1.5 

1 

1.5 

1 

1.5 

11 

72 

$366.30 

0.41% 

Asa  stint  Recotdei  Review 

8 

0.8 

7 

5.6 

143 

800.8 

1 

800.8 

9 

280 

$1 ,852.80 

43.22% 

Biiefei  Review  Annotation 

41 

3 

1 

3 

83 

249 

2 

498 

13 

360 

$27,633.59 

1.80% 

Brief  Piesentation  Vote 

44 

5 

11 

55 

1000 

59000 

2 

110000 

48 

120 

$29,655.56 

370.93% 

Totals 

100 

9.5 

23 

65.3 

111301 .5 

83 

$59,548.95 

186.91% 

Correlations 

Hours  to 

train 

Hours  to  train 

ccr  related 
to  Learning 

correlated  to 
Process 

time  in 

Instructions 

months 

0.955 

0.907 

"total  of  each  sub  process  instruction  x  portions  of 
hours  to  train 

Table  10  Overall  “as  is”  KVA  Analysis 


C.  “TO  BE”  INCREMENTAL  DESIGN  KVA  ANALYSIS 

With  the  base-line  model  we  created,  we  produced  our  incremental  “to-be”  model 
and  compared  the  two  for  any  differences  in  the  ROK  figures.  Additionally,  this  base  line 
model  can  be  used  to  compare  any  future  re-design  efforts,  including  a  radical  redesign  of 
the  process  such  as  using  the  five  vector  model  for  the  purposes  of  promotions.  As 
explained  previously,  our  “to-be”  incremental  design  is  envisioned  to  be  web-based  and 
thus  introducing  advanced  automation  and  accessibility  options  into  the  process.  A  major 
benefit  we  foresee  is  eliminating  redundant  or  repetitive  sub-processes  such  as  the  data 
download  stage  prior  to  each  tank  session.  Figure  8  shows  the  redesigned  or  incremental 
process  model. 

There  is  one  major  difference  between  the  “to-be”  and  “as-is”  model.  In  the  “to- 
be”  model,  we  completely  eliminated  the  sub-process  for  the  data  being  manually 
downloaded  to  the  tank.  Although  this  has  only  really  eliminated  two  process 
instructions,  we  show  in  our  spreadsheet  analysis  that  this  equates  to  a  significant  time 
reduction  in  the  overall  process. 
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The  following  is  a  list  of  the  “to-be”  sub-processes  with  their  respective  process 
instructions: 

•  IT  Review 

-  Process  instructions  remain  the  same  as  the  “to-be”  model. 

•  Assistant  Recorder  Review 

-  Process  instructions  remain  the  same 

•  Briefer  Data  Review  and  Formatted  for  Presentation 

-  Process  instructions  remain  the  same 

•  Brief  Presentation/V  ote 

-  Briefer  displays  and  orally  reviews  annotated  data. 

-  All  members  visually  compare  record  with  peer  performance  aggregate 
baseline  data. 

-  All  members  cast  vote  for  each  record. 


For  the  scope  of  our  POC  model,  the  sub-processes  and  process  instructions  for 
the  incremental  design  are  essentially  the  same  as  the  “as-is”  model  until  after  the  Briefer 
Review/ Annotation  sub-process.  However,  with  the  current  work  being  done  through  the 
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Navy’s  personnel  and  manpower  modernization  effort,  it  is  possible  for  all  of  the  sub¬ 
process  to  be  significantly  reduced,  thus  allowing  for  much  higher  ROK  figures  in  the 
future.  For  the  scope  of  this  project  however,  we  have  decided  to  limit  the  re-design  to 
only  those  sub-processes  that  would  be  directly  affected  by  our  POC  model.  The 
following  table  (Table  1 1)  lists  the  specific  numerical  difference  in  the  ROK  analysis  for 


the  “to-be”  model: 


VISUALIZATION  -  INCREMENTAL 

SELECTION  BOARD  PROCESS 

(PER  1000  RECORDS) 

Learning  time 

Value  added 

Persons 

Time 

X  #of 

Fired 

X  #  of 

=  Weighted 

Process 

Execution  time 

Hourly  rate 

Process  Costs 

ROK*** 

(months) 

(hours  to  train*) 

Involved 

Subtotal 

Times  Fired* 

Subtotal 

Inte  rations 

Value  Added 

Instructions  (uwt) 

(hours) 

($) 

($) 

<%) 

4 

0.2 

1 

0.2 

1 

0.2 

6 

1.2 

2 

6 

$6.78 

$40.68 

2.95% 

6 

0.5 

3 

1.5 

1 

1.5 

1 

1.5 

11 

72 

$5.09 

$366.48 

0.41% 

11 

0.8 

7 

5.6 

143 

800.8 

1 

800.8 

9 

280 

$6.62 

$1,853.60 

43.20% 

35 

3 

1 

3 

83 

249 

2 

498 

13 

240 

$76.76 

$18,422.40 

2.70% 

44 

5 

11 

55 

1000 

55000 

2 

110000 

48 

60 

$247.13 

$14,827.80 

741.85% 

100 

9.5 

23 

65.3 

111301.5 

83 

$35,510.96 

313.43% 

Hours  to 
train 

correlated  to 
Learning 
time  in 
months 

Hours  to  train 
correlated  to 
Process 
Instructions 

0.984 

0.907 

Table  1 1  Overall  “to-be”  KVA  analysis 


D.  COMPARATIVE  ANALYSIS 

The  KVA  methodology  offers  a  means  in  which  to  objectively  measure  the  value 
of  knowledge  assets  deployed  in  core  processes.  The  value  measurement  of  knowledge 
embedded  in  an  organization’s  core  processes,  technology,  and  employees  is 
accomplished  by  determining  what  the  organization’s  Return  on  Knowledge  (ROK)  is. 
The  numerator  of  the  ratio  represents  the  percentage  of  the  revenue  or  sales  dollar 
allocated  to  the  amount  of  knowledge  required  to  complete  a  given  process  successfully, 
in  proportion  to  the  total  amount  of  knowledge  required  to  generate  the  organization’s 
total  outputs.  The  denominator  of  the  ratio  is  the  cost  to  execute  the  process  knowledge. 
By  comparing  the  “as-is”  figure  in  Table  12  to  the  “to-be”  model  in  Table  13,  we  see  a 
significant  difference  in  the  ROK  figures.  The  “to-be”  model  is  a  close  approximation  of 
the  redesigned  web-enabled  promotion  process  to  include  the  enhanced  OSR/PSR 
graphical  display. 
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Process 

Execution  time 

Piocess  Costs 

ROK*** 

Instructions  juwt) 

(hours) 

($) 

(%) 

2 

6 

$40.70 

2.95% 

11 

72 

$366.30 

0.41  % 

9 

280 

$1 ,852.80 

43.22% 

13 

360 

$27,633.59 

1.80% 

48 

120 

$29,655.56 

370.93% 

83 

$59,548.95 

186.91  % 

Table  12 

“As-is”  ROK  figures 

Process 

Execution  time 

Process  Costs 

ROK*** 

Instructions  (uwt) 

(hours) 

_ m _ 

_ m _ 

1  111 

2 

6 

$40  68 

2.95% 

11 

72 

$366.48 

0.41  % 

9 

280 

$1 ,853.60 

43.20% 

13 

240 

$18,422.40 

2.70% 

48 

60 

$14,827.80 

741.85% 

83 

$35,510.96 

313.43% 

Table  13  “To-be”  ROK  figures 

As  evident  from  the  figures,  the  ROK  does  not  significantly  change  until  we 
change  the  later  sub-processes  of  Briefer  Review  and  the  Brief/Presentation/Vote.  In  the 
last  case  we  can  see  that  with  the  new  process,  the  ROK  in  last  sub-process  has  doubled 
from  370.93  percent  to  741.85  percent.  In  other  words,  the  amount  of  knowledge 
executed  during  that  time  frame  (knowledge  is  a  surrogate  for  the  process  outputs 
measured  in  common  units)  is  significantly  higher  in  the  Brief/Presentation/Vote  sub¬ 
process  with  the  “to-be”  incremental  design. 

Does  the  overall  increase  in  ROK  from  186.91  percent  to  313.43  percent  support 
our  principle  research  questions? 

•  Can  data-information  visualization  best  practices  be  embedded  in  information 
technology  in  order  to  improve  the  overall  effectiveness  and  objectiveness  of 
the  selection  board  process? 

•  Will  integrating  web-based  technology  with  the  visualization  techniques 
reduce  the  time  required  to  prepare,  brief,  and  make  decisions  in  the  promotion 
process? 
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These  questions  are  really  about  improving  the  effectiveness  and  efficiency  of  the 
decision  making  process.  Effectiveness  in  decision  making  is  focused  on  what  should  be 
done,  whereas  efficiency  is  focused  on  how  we  should  do  it.  George  M.  Marakas,  in  his 
book  Decision  Support  Systems  in  the  21st  Century,  explains  the  difference  between  the 
two  in  the  context  of  how  a  decision  support  system  contributes  to  each: 

Effectiveness 

•  Easier  access  to  relevant  infonnation. 

•  Faster  and  more  efficient  problem  recognition  and  identification. 

•  Easier  access  to  computing  tools  and  proven  model  to  compute  choice  criteria. 

•  Greater  ability  to  generate  and  evaluate  large  choice  sets. 

Efficiency 

•  Reduction  in  decision  costs. 

•  Reduction  in  decision  time  for  same  level  of  detail  in  the  analysis. 

•  Better  quality  in  feedback  supplied  to  the  decision  maker. 

By  strictly  comparing  the  KVA  ROK  figures  of  the  before  and  after  processes,  we 
can  easily  see  that  the  second  question  regarding  efficiency  has  been  improved.  By  using 
a  fully  web-enabled  system,  it  would  reduce  process  instructions  involved  in  the  later 
sub-processes  which  would  result  in  an  overall  more  efficient  process.  Further,  we  can 
surmise  that  by  using  our  approach,  the  execution  time  in  hours  would  decrease  from  360 
hours  in  the  Briefer  Review/ Annotation  phase  to  240  hours  for  every  1000  records.  The 
big  difference,  however,  was  reducing  execution  time  in  the  Brief/Presentation/Vote 
phase  from  120  hours  to  60  hours.  This  one  improvement  resulted  in  the  dramatic 
increase  in  ROK.  We  believe  that  the  data  visualization  techniques  would  help  shorten 
the  process  instruction  involved  in  the  “voting”  process  due  to  its  powerful  ability  to 
convert  data  objects  into  information  so  quickly. 

In  the  next  chapter  we  will  introduce  our  POC  prototype  web  site  that  will  shows 
how  we  can  further  increase  the  effectiveness  by  having; 

•  easier  access  to  relevant  information 

•  faster  and  more  efficient  problem  recognition  and  identification 

•  easier  access  to  computing  tools  and  proven  models  to  compute  choice  criteria 

•  provide  a  greater  ability  to  generate  and  evaluate  large  choice  sets 
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V.  VISUAL  ONLINE  PROMOTIONS  SYSTEM  (VOPS) 


A.  OVERVIEW 

In  this  chapter,  we  will  explain  the  Visual  Online  Promotions  System  (VOPS),  the 
system  we  developed  to  address  the  thesis  objectives.  We  begin  by  explaining  the 
different  security  approaches  we  considered  and  describe  the  construction  of  our 
information  technology  solution.  The  prototype  helps  the  reader  understand  our  redesign 
vision  and  supports  the  research  questions  of  this  thesis. 

B.  SECURITY  APPROACH 

The  current  promotion  system  consists  of  the  Electronic  Military  Personnel 
Records  Management  System  (EMPRS)  that  encompasses  the  Primary  Input/Output 
system  (PRIO),  the  Fitness  Report/Evaluation  system  (FITEV),  and  Selection  Board 
Module  (SELBD).  These  modules  work  together  to  create  a  system  that  allows  operators 
to  manage  member  records  from  the  desktop.  All  these  systems  are  internal,  meaning 
they  can  only  be  accessed  via  the  interfaces  at  the  Navy  Personnel  Command  in 
Millington  Tennessee.  The  advantages  to  this  approach  include  increased  security  and 
local  administration  of  the  intricate  information  systems. 

The  prototype  we  designed  is  intended  to  provide  access  through  the  Internet, 
using  encryption  and  password  protected  access.  The  advantages  to  having  the  system 
accessible  from  anywhere  a  person  has  Internet  connectivity  benefits  not  only  the  board 
members,  who  in  the  future  will  have  the  ability  to  peruse  records  from  their  own  station, 
but  also  the  promotion  candidate  who  can  access  the  system  and  see  what  information 
about  him  or  her  is  available,  current,  and  seen  by  the  board.  This  web-enabled  concept 
introduces  another  layer  of  security  concerns  and  requirements  beyond  the  scope  of  this 
thesis  but  we  designed  our  system  assuming  these  technical  and  procedural  concerns  will 
be  researched  and  implemented  separately. 

The  Visual  Online  Promotions  System  (VOPS)  may  be  implemented  using  either 
a  closed  intranet  or  public  Internet  approach,  but  we  designed  our  prototype  to  simulate 
access  via  the  Internet  because  we  believe  that  future  implementations  will  eventually  use 
a  web-enabled,  virtual  board  system  using  some  form  of  collaborative  technology. 
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C.  WEB  APPLICATION  PROTOTYPE  DEVELOPMENT 

To  construct  the  VOPS,  we  used  a  variety  of  software  components.  Specifically, 
Table  14  identifies  and  describes  the  purpose  of  each  software  component. 


Maker 

Software  Title 

Version 

Purpose 

Macromedia 

Dreamweaver  MX 

6.0 

Design  and  draft  static 
HTML  pages  and 
dynamic  ASP  pages 

Microsoft 

Access 

XP 

Create  a  database 
containing  the 
promotion  data 

Microsoft 

Internet  Explorer 

6.0 

View  completed  web 
pages  and  test 
application 

Microsoft 

Internet 

Infonnation  Server 

6.0 

Localized  server 
software  to  enable 
Dreamweaver  to  display 
dynamic  web  pages 
(connects  web  page, 
database,  and  browser) 

Tal 

ble  14  Software  solutions  usee 

l  to  develop  VOPS  prototy 

1.  Overview  of  3-Tier  Architecture 

While  the  scope  of  this  thesis  is  focused  on  streamlining  the  Naval  promotion 
system  using  a  notional  solution,  the  actual  solution  will  likely  involve  some  form  of  3- 
tier  architecture  so  we  believe  a  concept  overview  is  warranted. 

For  web  applications  such  as  what  we  are  proposing,  a  3-tier  architecture  is 
currently  the  best  solution.  It  consists  of  a  client  tier,  or  presentation  layer,  where  the  user 
sees  the  information.  The  second  tier  is  the  application  tier,  or  business-rule  layer,  and 
protects  the  data  from  direct  access  by  the  clients.  The  third  tier,  or  data  layer,  is  simply 
where  the  data  is  stored  and  is  accessed  via  the  application  layer.  The  first  two  tiers  are 
normally  housed  within  the  client’s  machine  with  the  data  layer  accessed  through  a 
network. 

Because  the  3 -tier  architecture  protects  the  data  and  provides  a  robust  design, 
creating  the  VOPS  using  this  popular  and  powerful  architecture  scheme  provided  the  best 
solution  for  our  design. 
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Using  the  software  tools  in  Table  14,  we  set  out  to  design  the  VOPS  with  a 
database,  static  web  pages,  and  dynamic  web  pages  using  the  Active  Server  Page  (ASP) 
technology.  By  creating  a  notional  Access  database  and  designing  web  pages  that 
dynamically  query  the  data,  we  were  able  to  implement  a  rapid  prototype  as  a  Proof  Of 
Concept  (POC). 

2.  The  Notional  Database 

We  designed  a  Microsoft  Access  database  to  house  the  VOPS  data  that  we  would 
query  from  the  web  pages.  Figure  9  shows  the  schema  we  designed  to  normalize  the  data 
and  to  provide  the  necessary  query  capabilities  to  query  a  variety  of  data. 


Figure  9.  Schema  of  the  VOPS’s  Notional  Database 


Each  block  represents  a  table  of  information  and  the  adjoining  lines  are 
relationships  between  the  tables,  hence  a  “relational”  database.  By  isolating  the  data  in 
this  way,  updates  and  maintenance  can  be  accomplished  cleanly  without  the  problems 
associated  with  having  redundant  information  in  different  locations. 
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It  is  important  to  note  that  the  tables  are  never  directly  accessed  but  only  via  a  set 
of  queries  designed  to  provide  subsets  of  the  data  for  the  requested  information,  as  seen 
in  Figure  10.  Note  also  that  horizontal  rows  contain  a  record,  or  collection  of  data  on  one 
person.  The  vertical  columns  are  the  individual  data  fields  for  each  record.  Both  queries 
and  tables  are  organized  in  this  manner. 


Figure  10.  The  Assignments  Query 


We  understand  that  Microsoft  Access,  while  a  powerful  prototyping  tool,  is  not 
scalable  for  enterprise-level  solutions  and  therefore  the  actual  operational  VOPS  would 
require  a  more  robust  database.  Nevertheless,  the  concepts  are  the  same.  The  data  will  be 
stored  in  a  database  and  accessed  by  the  design  architecture’s  middle  tier  on  its  way  to 
the  presentation  layer.  For  this  reason,  we  will  not  delve  too  deeply  into  the  mechanics  of 
Access  or  the  notional  database  we  created. 
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D.  THE  VOPS  PROTOTYPE 

Our  original  intent  for  the  VOPS  prototype  was  to  mimic  the  actual  Naval 
Personal  Command’s  web  site  design  and  navigation  to  the  point  where  we  introduce  our 
new  procedures.  To  that  end,  we  designed  our  pages  as  a  scaled  down  version  of  the 
actual  Personal  Command’s  web  site. 

On  the  BUPERS  homepage  (Figure  1 1),  a  link  takes  a  user  to  the  Selection  Board 
page  where  all  active  boards  are  listed  and  hyperlinks  take  the  user  to  the  particular  board 
he  or  she  is  interested  in.  (Figure  12). 


Figure  1 1 .  Simulated  homepage  for  BUPERS 
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Figure  12.  Selection  Board  Display  Page 

While  all  current  boards  are  listed,  only  boards  that  have  information  will  have 
links.  The  purple  link  in  figure  12  indicates  a  followed  link  which  takes  the  user  to 
Figure  13. 
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B00 


H  http://localhost/Visual/fyboards.asp  -  Microsoft  Internet  Explorer 


File  Edit  View  Favorites  lools  Hdp 

O  Back  O  3  3  fi)  Links  Google  Capt  G  a  MSN  ^NPSmail  NPS  Marines  BB  EB  41  USMC 


Address  :4)  h  ttp :  //localhostA'isual/fyboards .  asp 


NAV Y  PERSONNEL 
COMMAND 


9  m 


iLPOM 

ir^  Naval 


Bureau  of 

Naval  Personnel 


Welcome  to  the  new  improved  Visual  Online  Promotion  System  (V OPS). 


ACTIVE  DUTY  0  5  LINE  BOARD 
Board  ID#  is  230 

THE  BOARD  HAS  ADJOURNED 
Click  here  to  login  as  a  Board  Member 


FY  2004 

FY  2003 

Eligible  LCDRs  updated  08  Feb  03 

Eligible  LCDRs 

Membership 

Precept  Supplemental  Guidance 

Release  Process 

Membership 

Precept 

Release  Process 

Select  Message 

Selection  Board  Statistics 

Promotion  Pav  Increment  Plan 

DUE  TO  THE  SEVER  WEATHER  CONDITIONS  IN  THE  NORTHEAST  AND  SOME 

4} 


O  Trusted  sites 


Figure  13.  Individual  Board  Display  Page 


So  far,  we  have  not  introduced  any  changes  to  the  current  website.  In  our 
prototype,  anyone  can  access  the  pages  we  have  seen  thus  far;  it  is  only  at  Figure  13  that 
we  branch  off  to  provide  dual  capability  for  the  candidate  and  the  board  member. 

In  Figure  13,  a  board  member  can  access  the  promotion  board  pages  by  clicking 
on  the  “Click  here  to  login  as  a  board  member”  link  which  will  take  him  or  her  to  Figure 
14.  Regardless  of  the  security  approach  adopted,  this  log  in  page  will  provide  a  level  of 
security  to  the  process  necessary  to  protect  the  promotion  selection  process. 
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Figure  14.  VOPS  Board  Member  Log  In  Screen 

Once  the  user  has  been  identified  and  verified,  the  system  displays  only  the 
records  he  or  she  is  assigned  (Figure  15).  The  list  of  candidates  will  differ  from  board 
member  to  board  member  depending  on  the  individual  records  assigned  to  each.  With  the 
click  of  a  link,  the  board  member  has  access  to  each  candidate’s  Officer  Summary 
Record  (OSR,  Figure  16)  Cover  Page  and  Performance  Summary  Record  (PSR,  Figure 
17)  which  are  both  dynamic,  web-enabled  versions  of  the  documents  currently  used  to 
display  the  candidate’s  history.  We  believe  that  providing  a  layout  similar  to  what  has 
been  traditionally  provided  was  necessary  for  procedural  continuity  so  that  the  board 
members  can  access  detailed  information  for  their  pre-tank  record  review. 


62 


Figure  1 5 .  Candidate  Record  Display  for  Individual  Board  Member 
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Figure  16.  VOPS  OSR  Cover  Page 
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Figure  17.  VOPSPSRPage 


After  clicking  on  the  “Fitreps”  link  in  Figure  17,  the  user  is  taken  to  a  dynamic 
page  that  displays  summary  performance  marks  for  the  candidate  in  the  same  manner  the 
tank  currently  sees  the  information.  Because  the  reviewer’s  needs  differ  slightly  from  the 
tank  needs,  this  traditional  view  of  information  is  necessary  for  the  reviewer.  Because  the 
tank  members  benefit  from  summary,  graphical  information  rather  than  raw  data,  we 
redesigned  the  fitness  report  data  to  take  advantage  of  the  visual  techniques  discussed  in 
chapter  III  (Figure  18). 
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Figure  18.  Dynamic  Fitness  Report  Overview  Graph 

In  our  design,  the  data  is  normalized  so  that  a  reporting  senior’s  (RS)  grading 
behavior  is  neutralized.  For  example,  if  a  candidate  performs  extremely  well  at  his  first 
duty  station,  but  because  his  reporting  senior  tends  to  grade  everyone  toward  the  bottom 
of  the  performance  scale,  the  candidate  might  receive  a  grade  of  3.0  which  is  above 
average  for  that  reporting  senior.  At  his  next  command,  the  candidate  might  do  very 
poorly  but  the  reporting  senior  grades  high  with  a  result  of  a  4.0  which  translates  to  a 
below  average  mark  from  that  particular  reporting  senior.  Looking  at  these  two  grades 
without  normalizing,  a  board  member  will  interpret  that  the  candidate  performed  poorly 
at  his  first  command  and  better  at  his  second  command:  exactly  the  opposite  of  the 
reality. 

For  this  reason,  we  normalized  the  data,  taking  into  account  the  grading  history  of 

each  reporting  senior.  In  Figure  18,  a  grade  of  1.0  is  the  exact  average  of  the  reporting 

senior  so  anything  above  that  line  indicates  superior  performance,  and  where  anything 

below  1.0  indicates  poor  performance,  relative  to  the  reporting  senior  grading  criteria.  In 
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the  design  of  these  graphics,  we  tried  to  incorporate  the  important  principles  of  graphical 
integrity  as  spelled  out  in  chapter  three.  “Graphics  must  not  quote  data  out  of  context  . . . 
the  representation  of  numbers,  as  physically  measured  on  the  surface  of  the  graphic  itself, 
should  be  directly  proportional  to  the  numerical  quantities  represented”  (Tufte,  2001). 

Similar  to  the  fitness  report  display,  we  designed  a  page  that  dynamically  pulls 
award  information  and  displays  it  to  the  reviewer.  Figure  19  is  an  example  of  the  new 
awards  page  and  shows  the  board  members  at  a  glance  not  only  the  awards  the  candidate 
received,  but  the  relative  importance  of  each  award,  indicated  by  the  height  of  the  bar:  the 
higher  the  award  precedence,  the  taller  the  bar.  In  this  way,  the  board  spends  less  time 
correlating  the  different  awards  and  can  see  when  each  award  was  earned. 


Figure  19.  Dynamic  Awards  Graph 

By  providing  the  user  with  individual  charts,  the  prototype  enables  both  the 
reviewer  and  the  board  members  with  specific  information  concerning  each  candidate. 
But  the  vision  of  the  VOPS  was  to  aid  the  user  in  quickly  and  efficiently  interpreting  a 
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range  of  promotion  factors.  With  this  in  mind,  the  system  provides  a  combined  chart 
(Figure  20)  that  offers  a  comparative  correlation  over  a  common  timeline.  With  this 
disparate  data  collected  into  one  chart,  the  board  members  can  easily  achieve 
comparative  analysis  at  a  glance.  As  Tufte  writes, 

We  envision  infonnation  in  order  to  reason  about,  communicate, 
document,  and  preserve  that  knowledge  -  activities  nearly  always  carried 
out  on  two-dimensional  paper  and  computer  screen.  Escaping  this  flatland 
and  enriching  the  density  of  data  displays  are  the  essential  tasks  of 
information  design  (Tufte,  1990). 

Using  this  concept  of  information  density,  each  layer  extracts  data  and  translates 
it  into  a  graphical  output  on  the  common  timeline.  A  briefer  has  the  option  to  choose 
which  graphs  he  or  she  wants  to  display  and  also  has  the  capability  to  drill  down  for  more 
detail  by  simply  clicking  on  the  graphs. 

The  system  also  provides  tools  for  the  briefer  to  highlight  important  aspects  of  the 
candidate’s  career  by  using  the  mouse  to  “draw”  on  the  screen. 

The  idea  of  the  display  capability  is  to  provide  the  briefer  with  a  set  of  tools  he  or 
she  can  choose  from  while  preparing  the  presentation  prior  to  the  tank  sessions.  The 
precept  could  dictate  common  presentation  rules  but  within  those  boundaries,  the  briefer 
has  the  capability  to  organize  a  specialized  brief,  complete  with  highlights  and  optional 
displays,  for  each  record. 

In  Figure  20,  the  top  line  is  an  overview  of  the  candidate’s  sea  duty  (blue  bar) 
versus  shore  duty  (brown  bar).  The  next  layer  is  the  candidate’s  promotion 
recommendations  drawn  from  the  PSR  information  which  correlates  to  the  performance 
recommendations,  or  the  “fitrep”  overview,  displayed  on  the  third  tier.  Considered 
individually,  the  layers  convey  trend  lines  over  time.  Taken  together,  the  board  members 
can  easily  correlate  different  aspects  of  the  candidate’s  record  over  a  common  timeline 
and  can  more  easily  make  associations  to  certain  data. 

The  awards  data,  similar  to  the  data  displayed  in  Figure  19,  appears  on  the  fourth 
level  where  we’ve  added  a  color  coding  system  that  aids  in  award  type  differentiation 
where  the  eye  can  easily  follow  a  vertical  line  that  associates  the  award  with  a  specific 
time  period. 
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The  final  layer  shows  the  duty  assignment  overview  as  a  baseline  where  the  board 
can  get  a  common  understanding  of  the  unit  the  candidate  was  working  at  in  relation  to  a 
time  dimension.  Additionally,  the  system  provides  the  candidate’s  billet  assignment 
above  the  command  he  or  she  was  assigned  to.  This  information  is  vital  when  the  board 
sees  the  candidate’s  display  because  they  are  able  to  match  the  type  and  duration  of  a 
specific  billet  with  corresponding  timeframes. 


Figure  20.  Consolidated  Promotion  Data  Page 
In  our  first  iteration  of  the  prototype,  we  did  not  provide  the  duty  assignment 
information.  After  feedback  from  the  Naval  Personnel  Command,  we  discovered  a  need 
to  further  specify  not  only  the  unit  that  the  candidate  belonged  to,  but  what  his  or  her  job 
title  was  during  the  tour.  We  therefore  designed  an  improved  duty  layer  that  showed  both 
the  command  and  duty  assignment  over  the  common  timeline,  as  seen  at  the  bottom  of 
Figure  20. 
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E.  SUMMARY 

Regardless  of  the  approach  eventually  adopted,  the  VOPS  prototype  can  be 
implemented  via  an  Internet  or  Intranet  solution.  By  using  three-tier  architecture,  the 
system  provides  a  maintainable,  stable,  and  robust  capability  and  enables  the  users  to 
quickly  and  simply  display  a  large  amount  of  promotion  data  in  a  superior  interface.  The 
improved  presentation  takes  advantage  of  proven  techniques  outlined  in  chapter  III  and 
provides  both  the  briefers  and  board  members  with  powerful  cognitive  tools  to  analyze 
and  compare  the  different  promotion  criteria  for  each  candidate.  The  end  result  is  less 
mental  computation  effort  required  of  the  board  members  and  a  quicker  decision  cycle. 
Together,  these  factors  increase  the  possibility  of  quality  decisions,  translating  in  a  better 
promotion  system. 

Our  ROK  analysis  has  proven  that  by  using  a  Web-enabled  system  and 
graphically  displaying  the  information  it  would  make  the  process  more  efficient  saving 
time  and  money  in  each  board  session.  In  order  to  answer  the  first  question  of  whether  or 
not  the  new  process  is  more  effective  or  more  objective  we  would  propose  using  a  side  by 
side  analysis  where  the  new  system  is  used  in  conjunction  with  the  old  system  in  an 
actual  board.  We  would  recommend  that  this  board  be  a  smaller  administrative  board 
where  the  participant  board  members  would  have  the  time  and  leeway  to  respond  with 
insights  and  suggestions.  By  surveying  the  board  members  immediately  after  the  tank 
session  we  could  gain  immediate  and  helpful  insights  as  to  weather  or  not  the  graphical 
displays  assist  in  perceiving  and  correlating  the  quantitative  information.  They  could 
provide  invaluable  information  as  to  how  to  improve  the  test  system  as  well.  By 
surveying  multiple  boards  and  making  the  suggested  improvements  along  the  way  it 
would  be  possible  to  acquire  empirical  feedback  necessary  to  determine  whether  or  not 
the  proposed  system  contributes  or  detracts  from  increasing  the  effectiveness  and 
objectiveness  of  the  overall  process.  We  believe  that  by  reducing  the  preparation, 
briefing,  and  voting  times  for  the  briefers’,  we  can  ultimately  reduce  some  of  the 
emotional  and  physical  strain  internal  to  multiple  decisions  in  a  confined  time  period. 
This  improvement  may  well  translate  to  better,  more  objective,  and  ultimately  more 
effective  and  objective  decisions. 
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VI.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

This  thesis  examines  the  Naval  Promotion  system  and  how  current  board 
procedures  display  information  to  the  decision-makers.  By  examining  proven  data 
visualization  methods  and  integrating  information  technology  into  the  system,  we  explore 
the  benefits  realized  by  re-designing  the  procedures  that  currently  require  the  board 
members  to  process  raw  data  into  useful  information. 

This  chapter  provides  the  conclusions  drawn  from  the  research  .  It  also  provides 
recommendations  for  further  study  in  the  areas  of  IT  implementation  and  visualization 
techniques. 

B.  CONCLUSIONS 

The  Naval  selection  board  process  has  embedded  procedures  that  have  sufficed 
for  years.  By  observing  their  methods,  we  discovered  the  possibility  to  make  the  working 
system  even  better  based  on  modem  IT  and  visualization  theories.  We  concluded  that 
there  is  no  need  for  a  drastic  overhaul  of  the  entire  system  but  rather  an  incremental 
upgrade  incorporated  within  their  current  procedures.  Specifically,  our  thesis  concludes 
that  a  relatively  small  investment  in  IT  will  result  in  significant  returns  for  many  aspects 
of  the  existing  framework. 

1.  Effectiveness  and  Objectiveness  of  the  Selection  Board  Process 

Our  primary  thesis  question  asked  if  a  data-information  visualization  best 
practices  can  be  embedded  in  information  technology  in  order  to  improve  the  overall 
effectiveness  and  objectiveness  of  the  selection  board  process.  Using  the  Knowledge 
Value  Added  (KVA)  methodology  as  a  metric,  we  concluded  that  significant  gains  are 
possible  using  data-to-information  visualization  best  practices  embedded  in  information 
technology.  Our  results  show  a  126.52  percent  increase  in  ROK.  by  introducing  our 
recommendations.  We  can  conclude  from  the  increase  in  ROK  that  the  efficiency  of  the 
“to-be”  process  would  be  significantly  improved  from  the  changes.  By  using  a  fully 
web-enabled  system,  it  would  reduce  process  instructions  involved  in  the  sub  -  processes 
that  take  place  later  stages  of  the  boards  overall  process  which  would  result  in  an  overall 
more  efficient  process.  Further,  we  can  sunnise  that  by  using  our  approach,  the 
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execution  time  in  hours  would  decrease  by  120  hours  for  every  1000  records  briefed  in 
the  Briefer  Review/Annotation  phase.  Additionally,  the  “to-be”  model  showed  a 
reduction  of  60  hours  per  1000  records  in  the  Brief/Presentation/Vote  phase.  Further,  we 
believe  that  by  implementing  the  data  visualization  techniques  introduced  in  this  thesis  it 
would  shorten  the  process  instruction  involved  in  the  “voting”  process  even  more  due  to 
its  powerful  ability  to  convert  data  objects  into  information  so  quickly. 

Long  tenn  effects  could  include  an  increase  in  retention  rates  due  to  the 
enhancement  of  personnel  satisfaction  attributed  to  a  more  effective  determination  of  the 
future  promotion  or  assignment  of  Navy  personnel.  Unfortunately  there  is  no  way  of 
knowing  or  accurately  measuring  the  validity  of  this  assumption.  However,  it  is 
reasonable  to  conclude  that  by  decreasing  the  strain  on  the  board  members  and  the  time  it 
takes  to  mentally  assimilate  the  data  required  to  make  a  timely  decision  that  this  would 
ultimately  result  in  an  overall  increase  in  the  boards  effectiveness  especially  in  the  later 
stages  of  the  board  when  board  members  are  more  susceptible  to  burn  out  or  fatigue. 

The  design  of  these  displays  should  strive  for  uniformity  in  graphics  and 
assurance  that  the  representation  accurately  depicts  the  target  numbers.  We  believe  that 
the  objectivity  of  the  current  system  will  be  maintained  or  enhanced  by  designing  the 
graphical  display  in  such  a  manner  that  they  accurately  portray  the  information  that  is 
given.  The  ultimate  design  of  the  system  should  keep  in  mind  Tufte’s  guidance  that  the 
representation  of  numbers,  as  physically  measured  on  the  surface  of  the  graphic  itself, 
should  be  directly  proportional  to  the  numerical  quantities  represented.  Additionally,  to 
gain  the  most  objective  results  a  clear,  detailed,  and  thorough  labeling  should  be  used  to 
defeat  graphical  distortion  and  ambiguity.  Write  out  explanations  of  the  data  on  the 
graphic  itself.  Label  important  events  in  the  data. 

2.  Time  Reduction 

We  concluded  that  integrating  web-based  technology  with  the  visualization 
techniques  will  reduce  the  time  required  to  prepare,  brief,  and  make  decisions  in  the 
promotion  process.  The  current  process  uses  a  proprietary  display  system  that  simply 
projects  pictures  of  paper-based  records  and  a  crude  interface  of  summary  data.  By 
offering  a  web  interface,  we  conclude  that  the  same  data  drawn  from  a  database  and 

displayed  in  graphical  fonns  or  organized  in  a  system  of  hyperlinks  will  reduce  the  time 
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required  of  all  participants.  The  preparer  spends  less  time  correlating  the  data  for  the 
board  members  which  shortens  his  or  her  brief.  More  importantly,  the  system  enables  the 
board  members  to  make  quicker  decisions  based  on  infonnation  more  easily  ingested. 

3.  Appropriate  Technologies 

Web-based  data  visualization  technologies  can  serve  as  an  enabler  to  this  problem 
but  the  question  remains  of  what  are  the  relevant  criteria  for  deciding  which  technologies 
are  the  most  appropriate. 

Based  on  our  research  and  experience  with  this  problem,  we  conclude  that  a  three- 
tier  architecture  is  most  appropriate.  The  robust  nature  and  the  current  maturity  of  the 
architecture  lends  itself  to  storing,  processing,  and  presenting  information  needed  for  the 
promotion  boards.  Using  this  layered  approach,  technology  advances  can  be  adopted  in 
each  layer  thus  providing  expandability  to  the  system  without  requiring  complete 
dependency  on  current  technology  maturity  in  the  entire  system. 

4.  User  Interface  Design 

What  is  an  appropriate  user  interface  design  for  this  decision  problem?  We 
believe  that  a  web-based  interface  provides  a  clear,  concise,  familiar  interface  for  the 
modem  board  member.  Using  graphs  generated  by  the  data,  the  board  members  can 
quickly  assimilate  the  data  and  make  a  decision.  As  the  members  become  accustomed  to 
the  fonnat,  time  required  to  locate  pertinent  data  decreases.  By  using  the  interface  most 
prevalent  in  personal  and  professional  computer  interaction,  we  take  advantage  of  a  built- 
in  skill  set  to  the  computer- savvy  members.  Even  for  those  less  computer-savvy,  the 
interfaces  are  intuitive  and  the  graphs  adhere  to  the  proven  theories  set  forth  by  Tufte, 
thus  benefiting  all  the  board  members  regardless  of  their  technical  background. 

C.  RECOMMENDATIONS  FOR  FURTHER  STUDY 

1.  Implementation  of  the  Theoretical  Study 

Within  the  next  year  is  an  optimal  timeframe  to  carry  forward  the  research 
outlined  in  this  thesis.  The  Military  Personnel  Command,  in  conjunction  with  the  Space 
and  Naval  Warfare  Systems  Infonnation  Technology  Center  (SITC),  developed  the 
Single  Integrated  Human  Resources  Strategy  (SIHRS).  The  goal  of  this  strategy  is  to 
support  the  Defense  Integrated  Military  Human  Resources  System  (DIMHRS).  DIMHRS 
is  the  Joint  program,  also  initiated  from  the  1996  Defense  Science  Board,  which  will 
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unify  human  resource  support  for  both  active  duty  and  reserve  members  of  all  four  of  the 
anned  services  (Feldmann,  2003).  Additionally,  at  this  time  Naval  Personnel  Command 
is  implementing  a  technical  up-grade  on  the  EMPRS  system  and  the  goal  is  that  it  will  all 
be  Web-based  in  the  near  future.  We  recommend  collaborative  involvement  with  NPC  to 
incorporate  our  theoretical  solution  into  the  evolving  upgrades.  This  thesis  provides 
academic  support  based  on  proven  visualization  methods  therefore  the  next  logical  step  is 
to  move  forward  with  NPC  for  the  implementation  phase.  Important  factors  to  take  into 
account  are  the  following: 

•  compatible  with  EMPRS 

•  compliant  within  the  NMCI  network  standards 

•  compatible  with  PeopleSoft  HRMS  applications 

2.  Five-Vector  Model  Incorporation 

Eventually  a  system  such  as  this  could  be  incorporated  to  display  the  Five  Vector 
Model  where  individuals  would  be  evaluated  for  promotion  and  advancement  throughout 
their  career  as  outlined  by  the  vector  model.  The  Five  Vector  Model  aligns  professional 
development  requirements  with  certifications  and  qualifications  both  military  and  civilian 
that  are  recognized  as  the  industry  standard.  Once  the  fleet  has  identified  the 
requirements,  subject  matter  experts  in  each  field  of  expertise  plot  them  on  the  vector 
model  where  an  individual  should  ideally  meet  those  milestones  in  their  career.  The 
model  also  considers  personal  development,  leadership  and  management,  and  individual 
performance.  By  using  the  measures  of  performance  in  the  Five  Vector  Model,  it  may 
eventually  be  possible  to  eliminate  the  correlation  of  OSR/PSR  data  in  the  promotion 
process  entirely. 

Implementing  our  design  using  the  model  as  a  baseline  template  is  more  of  a 
radical  redesign  but  something  that  PERS-8  has  a  definite  interest  in.  We  recommend 
having  one  vector  on  one  screen  (five  vectors  for  the  five  screens).  Using  this 
methodology,  the  board  would  not  even  have  to  use  the  OSR/PSR  data  but  rather  gather 
the  information  they  need  based  on  the  combination  of  visualization  techniques  outlined 
in  this  thesis  and  the  Five  Vector  Model.  Furthermore,  by  directly  extracting  the  data  for 
the  Five  Vector  Model  from  Naval  Personnel  Database  or  NSIPS  (Navy  Standard 
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Integrated  Personnel  System  which  will  replace  4  legacy  systems  used  in  the  Fleet),  a 
single  source  solution  will  be  possible. 

3.  Other  Services 

The  ideas  and  prototype  we  developed  for  this  thesis  are  portable  to  the  other 
services.  Minor  personalization  for  each  branch  would  be  necessary  but  implementing  the 
system  could  improve  all  the  board  operating  procedures.  We  recommend  research  into 
the  other  services’  promotion  processes  and  an  evaluation  of  each  to  determine  how  this 
thesis  could  improve  each  process.  We  feel  that  such  an  evaluation  would  reveal 
tremendous  possibilities  to  improve  any  and  all  promotion  boards. 

4.  Virtual  Board 

The  most  radical  recommendation  we  have  is  a  study  to  determine  if  a  virtual 
board  could  eventually  occur.  Due  to  the  rapidly  advancing  technologies  of  collaborative 
tools  and  database  utilities,  combined  with  high  bandwidth  and  reliable,  secure 
connections,  we  feel  that  the  Navy  could  save  a  large  amount  of  funds  by  conducting 
boards  in  a  disbursed,  collaborative  manner. 

Our  vision  is  to  have  board  members  log  on  via  secure  accounts  and  perform  their 
record  evaluations  online.  The  centralized  hub  could  take  care  of  the  administration  and 
logistics  but  massive  savings  could  be  realized  by  not  requiring  board  members  to  travel 
to  a  common  site  for  the  proceedings.  Whether  the  board  would  resemble  a  Video 
Teleconference  (VTC)  or  a  prerecorded  presentation  of  each  record  brief  are  some  of  the 
issues  we  recommend  for  further  study. 
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